Visual compatibility modeling is central to modern fashion recommendation systems. A key task is complementary item retrieval , where the goal is to identify a garment that harmonizes with a reference item, such as retrieving a compatible bottom for a given top. Recent generative approaches synthesize candidate garments to guide retrieval, but they either treat generation as an auxiliary signal or rely on computationally demanding architectures, limiting their practicality in large-scale deployments. In this work, we introduce GeCo , a generative–compositional framework that couples image synthesis and retrieval within an effective, two-stage design. In the first stage, a conditional GAN generates visually coherent bottom templates from top images; in the second stage, the generated template and top form a composed visual query for compatibility-based retrieval. This decoupled design avoids the heavy optimization pipelines used in prior generative approaches, resulting in stable training and low computational cost, while allowing the generated images to guide compatibility modeling. Experiments on three benchmarks, including the new FashionTaobaoTB dataset released with this work, show that GeCo offers competitive retrieval accuracy with a low memory footprint. Human evaluations further indicate that its generated items are perceived as realistic and stylistically compatible, supporting its suitability for practical, resource-constrained fashion recommendation scenarios. • GeCo : generative fashion compatibility model and complementary item retrieval. • GeCo uses a two-stage GAN under realistic hardware limits. • Human studies find its garments realistic and stylistically coherent. • FashionTaobaoTB and benchmarking codebase have been released.

Content recommender systems (RSs), which recommend content (e.g., news, videos) to users, can inadvertently facilitate the spread of misinformation (e.g., fake news, inaccurate content) due to their ignorance of content credibility. The widespread misinformation can lead to serious consequences, including public confusion and social unrest. Despite the pressing need to mitigate misinformation in content RSs, only a few studies have attempted to tackle this. Moreover, they generally require retraining the entire model from scratch, which is time-consuming and impractical for real-world applications. To this end, we propose MisEraser , a novel misinformation unlearning framework that effectively mitigates misinformation in content RSs without retraining. Specifically, MisEraser consists of three key components: (1) A misinformation disentanglement network , which effectively separates misinformation-specific information from item content while preserving misinformation-free information to maintain recommendation accuracy; (2) A misinformation fusion network , which fuses misinformation-specific signals from both users’ interaction sequences and the entire misinformation item set to enhance the comprehensiveness of misinformation unlearning; and (3) A misinformation erasing network , which effectively removes the fused misinformation-specific signals from trained recommendation models, enabling them to generate misinformation-suppressed and more responsible recommendations. Extensive experiments demonstrate the effectiveness of MisEraser in mitigating misinformation while maintaining high recommendation accuracy and efficiency.

Countless new machine learning models are published every year and are reported to significantly advance the state-of-the-art in top-n recommendation. However, earlier reproducibility studies indicate that progress in this area may be quite limited, due to widespread methodological issues, e.g., comparisons with untuned baseline models, creating an illusion of progress . In this work, we examine whether these problems persist in today’s research by attempting to reproduce nine SIGIR 2023 and 2024 recommendation algorithms based on Denoising Diffusion Probabilistic Models, a recent but rapidly expanding research area. Only 25% of reported results are fully reproducible and, since the original papers relied on weak baselines , they do not establish the superiority of diffusion models over state-of-the-art methods. In our controlled evaluations, well-tuned simpler baselines consistently exceed the diffusion-based models’ effectiveness reported in the original papers. Furthermore, we identify key mismatches between the characteristics of diffusion models and those of the traditional top-n recommendation task, raising doubts about their suitability for recommendation. Moreover, in the analyzed papers, the generative capabilities of these models are constrained to a minimum. Overall, our results call for greater scientific rigor and a disruptive change in the research and publication culture in this area.

The sequential recommendation problem has attracted considerable research attention in the past few years, leading to the rise of numerous recommendation models. In this work, we explore how Large Language Models (LLMs), which are nowadays introducing disruptive effects in many AI-based applications, can be used to build or improve sequential recommendation approaches. Specifically, we design three orthogonal approaches and hybrids of those to leverage the power of LLMs in different ways. In addition, we investigate the potential of each approach by focusing on its technical aspects and determining an array of alternative choices for each one. We conduct extensive experiments on three datasets and explore a large variety of configurations, including different language models and baseline recommendation models, to obtain a comprehensive picture of the performance of each approach. Among other observations, we highlight that initializing state-of-the-art sequential recommendation models such as BERT4Rec or SASRec with embeddings obtained from an LLM can lead to substantial performance gains in terms of accuracy. Furthermore, we find that fine-tuning an LLM for recommendation tasks enables it to learn not only the tasks but also the concepts of a domain to some extent. We also show that fine-tuning OpenAI GPT leads to considerably better performance than fine-tuning Google PaLM 2. Overall, our extensive experiments indicate a huge potential value of leveraging LLMs in future recommendation approaches. We publicly share the code and data of our experiments to ensure reproducibility. 1

Offline evaluation is the predominant method for scientific research in recommender systems, enabling the comparison of alternative recommendation approaches using pre-collected datasets and computational metrics without involving human participants. However, recent reproducibility studies reveal that many offline evaluations in the literature lack scientific rigor or adopt research practices that cast doubt on the validity of their findings. A particularly common and ultimately catastrophic flaw is the comparison of newly proposed machine learning models against untuned or poorly tuned baseline models. Combined with limited reproducibility, such practices raise serious concerns about the true progress achieved by increasingly complex recommendation algorithms. In this editorial, we argue for stronger methodological standards and summarize essential guidance and best practices for conducting rigorous offline evaluations of recommender systems. Accordingly, ACM Transactions on Recommender Systems will place increased emphasis on methodological rigor in all future submissions, with particular priority given to work that provides comprehensive reproducibility materials and clearly documents the tuning procedures used for baseline models.

The idea of calibrated recommendations is that the properties of the items that are suggested to users should match the distribution of their individual past preferences. Calibration techniques are therefore helpful to ensure that the recommendations provided to a user are not limited to a certain subset of the user’s interests. Over the past few years, we have observed an increasing number of research works that use calibration for different purposes, including questions of diversity, biases, and fairness. In this work, we provide a survey on the recent developments in the area of calibrated recommendations. We both review existing technical approaches for calibration and provide an overview on empirical and analytical studies on the effectiveness of calibration for different use cases. Furthermore, we discuss limitations and common challenges when implementing calibration in practice.

The RecSys Challenge 2025 promotes a unified approach to behavior modeling by introducing Universal Behavioral Profiles. These user representations encode essential aspects of past interactions and are designed for universal applicability across different downstream tasks, thereby promoting generalization across applications and addressing the need for portable and efficient recommender systems. The participants task was to create universal user embeddings from detailed e-commerce activity logs. These embeddings were then fed into a small neural network to predict customer behavior in subsequent timeframes. The provided challenge dataset was large and sparse, requiring innovative methods to leverage the available interaction data in an effective way. Overall, the challenge was highly attractive with 400 teams participating in the competition.

Pre-trained language models (PLMs) are widely used to derive semantic representations from item metadata in recommendation and search.In sequential recommendation, PLMs enhance ID-based embeddings through textual metadata, while in product search, they align item characteristics with user intent.Recent studies suggest task and domain-specific fine-tuning are needed to improve representational power.This paper challenges this assumption for e-commerce applications, showing that Generalist Text Embedding Models (GTEs), pre-trained on large-scale corpora, can guarantee strong zero-shot performance without specialized adaptation.Our experiments on popular e-commerce benchmarks demonstrate that GTEs outperform traditional and fine-tuned models in both sequential recommendation and product search.We attribute this to a superior representational power, as they distribute features more evenly across the embedding space.Finally, we show that compressing embedding dimensions by focusing on the most informative directions (e.g., via PCA) effectively reduces noise and improves the performance of specialized models.To ensure reproducibility, we provide our repository at https://github.com/sisinflab/GTE-Zero- Shot-Recsys.

Public service media (PSM) providers commonly face the challenge of balancing user engagement metrics and public value.In this case study, we report on the insights obtained at ARD, Germany's largest PSM provider, when investigating the effectiveness of different collaborative filtering techniques on their video-on-demand platform ARD Mediathek.While an offline evaluation indicated that a modern model based on a denoising auto-encoder might lead to the best prediction accuracy, A/B testing revealed that an item-based nearest-neighbor technique excelled both in terms of engagement and public value metrics.Our findings thus suggest that traditional, light-weight models should not be easily dismissed, given also their comparably limited resource requirements and light carbon footprint.To enable future research on this topic, we provide a real-world dataset with usage data from our platform.

Issues with reproducibility have been identified as a major factor hampering progress in recommender systems research. In response, researchers increasingly share the code of their models. However, the provision of only the code of the proposed model is usually not sufficient to ensure reproducibility. In many works, the central claim is that a new model is advancing the state of the art. Thus, it is crucial that the entire experiment is reproducible, including the configuration and the results of the considered baselines. With this work, our goal is to gauge the level of reproducibility in algorithms research in recommender systems. We systematically analyzed the reproducibility level of 65 papers published at a top-ranked conference during the last three years. Our results are sobering. While the model code is shared in about two thirds of the papers, the code of the baselines is provided only in eight cases. The hyperparameters of the baselines are reported even less frequently, and how these were exactly determined is not explained in any paper. As a result, it is commonly not only impossible to reproduce the full result tables reported in the papers, it is also unclear if the claimed improvements over the state of the art were actually achieved. Overall, we conclude that the research community has not reached the required level of reproducibility yet. We therefore call for more rigorous reproducibility standards to ensure progress in this field.

Over the past few years, researchers have explored the use of reinforcement learning (RL) for sequential recommendation problems. However, since RL techniques commonly target at optimizing long-term rewards, it is surprising that RL-based models are reported to be competitive with traditional supervised models when evaluated under the myopic next-item prediction protocol. A recent study suggests that reported performance gains of combining RL with supervised learning techniques, as done in the Self-Supervised Q-Learning (SQN) framework, may actually not come from learning an optimal policy, but that the RL component helps to learn embeddings that encode the users' past interactions. Given these observations, we aimed to reassess the performance of RL-enhanced sequential recommendations in the SQN framework. While we were able to reproduce the results reported in the respective papers, we found that properly-tuned supervised learning models like GRU4Rec substantially outperform the proposed RL-models from the literature. Our analyses furthermore revealed that there is a significant inconsistency in terms of evaluation protocols in the literature, and that the use of third-party implementations of existing models may lead to unreliable conclusions. Overall, still more research and alternative evaluation schemes seem required to fully leverage the power of RL for sequential recommendation tasks.

Lately, we have observed a growing interest in intent-aware recommender systems (IARS). The promise of such systems is that they are capable of generating better recommendations by predicting and considering the underlying motivations and short-term goals of consumers. From a technical perspective, various sophisticated neural models were recently proposed in this emerging and promising area. In the broader context of complex neural recommendation models, a growing number of research works unfortunately indicates that (i) reproducing such works is often difficult and (ii) that the true benefits of such models may be limited in reality, e.g., because the reported improvements were obtained through comparisons with untuned or weak baselines. In this work, we investigate if recent research in IARS is similarly affected by such problems. Specifically, we tried to reproduce five contemporary IARS models that were published in top-level outlets, and we benchmarked them against a number of traditional non-neural recommendation models. In two of the cases, running the provided code with the optimal hyperparameters reported in the paper did not yield the results reported in the paper. Worryingly, we find that all examined IARS approaches are consistently outperformed by at least one traditional model. These findings point to sustained methodological issues and to a pressing need for more rigorous scholarly practices.

The essence of calibration in recommender systems is to generate recommendations that match the distribution of a given user’s past preferences regarding certain item features—e.g., in terms of preferred genres in the case of movies—while preserving relevance. The user’s past preference distribution is usually derived by considering the features of all items that the user previously liked. However, the most common approach in the literature to derive this distribution has certain limitations. First, it does not consider that user preferences may change over time. Second, there are domains where the relevant item features are set-valued, e.g., a movie can have several genres. In such cases, existing calibration approaches may represent the true user’s preference distribution in a suboptimal way. In this work, we, therefore, propose two novel approaches to derive the preference distributions of users for the purpose of calibration. The first method allows us to decrease the relevance of possibly outdated preference information. The second method is an entropy-based approach, which aims to capture better the user’s true preferences toward certain item features. Extensive experimental evaluations on four distinct datasets confirm that the proposed techniques are more effective in reducing the level of miscalibration than the common state-of-the-art calibration approach.

Many modern online services feature personalized recommendations. A central challenge when providing such recommendations is that the reason why an individual user accesses the service may change from visit to visit or even during an ongoing usage session. To be effective, a recommender system should therefore aim to take the users’ probable intent of using the service at a certain point in time into account. In recent years, researchers have thus started to address this challenge by incorporating intent-awareness into recommender systems. Correspondingly, a number of technical approaches were put forward, including diversification techniques, intent prediction models, or latent intent modeling approaches. In this article, we survey and categorize existing approaches to building the next generation of Intent-Aware Recommender Systems (IARS). Based on an analysis of current evaluation practices, we outline open gaps and possible future directions in this area, which in particular include the consideration of additional interaction signals and contextual information to further improve the effectiveness of such systems.

introduction Free Access Share on Just AcceptedSpecial Issue on Responsible Recommender Systems Part 1 Authors: Lina Yao CSIRO's Data61 and University of New South Wales, Australia CSIRO's Data61 and University of New South Wales, Australia 0000-0002-4149-839XSearch about this author , Julian McAuley University of California, USA University of California, USASearch about this author , Xianzhi Wang University of Technology Sydney, Australia University of Technology Sydney, Australia 0000-0001-9582-3445Search about this author , Dietmar Jannach University of Klagenfurt, Austria University of Klagenfurt, Austria 0000-0002-4698-8507Search about this author Authors Info & Claims ACM Transactions on Intelligent Systems and TechnologyAccepted on April 2024https://doi.org/10.1145/3663528Online AM:15 June 2024Publication History 0citation5DownloadsMetricsTotal Citations0Total Downloads5Last 12 Months5Last 6 weeks5 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Publisher SiteeReaderPDF

Reinforcement learning serves as a potent tool for modeling dynamic user interests within recommender systems, garnering increasing research attention of late. However, a significant drawback persists: its poor data efficiency, stemming from its interactive nature. The training of reinforcement learning-based recommender systems demands expensive online interactions to amass adequate trajectories, essential for agents to learn user preferences. This inefficiency renders reinforcement learning-based recommender systems a formidable undertaking, necessitating the exploration of potential solutions. Recent strides in offline reinforcement learning present a new perspective. Offline reinforcement learning empowers agents to glean insights from offline datasets and deploy learned policies in online settings. Given that recommender systems possess extensive offline datasets, the framework of offline reinforcement learning aligns seamlessly. Despite being a burgeoning field, works centered on recommender systems utilizing offline reinforcement learning remain limited. This survey aims to introduce and delve into offline reinforcement learning within recommender systems, offering an inclusive review of existing literature in this domain. Furthermore, we strive to underscore prevalent challenges, opportunities, and future pathways, poised to propel research in this evolving field.

Sequential recommendation problems have received increasing attention in research during the past few years, leading to the inception of a large variety of algorithmic approaches. In this work, we explore how large language models (LLMs), which are nowadays introducing disruptive effects in many AI-based applications, can be used to build or improve sequential recommendation approaches. Specifically, we devise and evaluate three approaches to leverage the power of LLMs in different ways. Our results from experiments on two datasets show that initializing the state-of-the-art sequential recommendation model BERT4Rec with embeddings obtained from an LLM improves NDCG by 15-20% compared to the vanilla BERT4Rec model. Furthermore, we find that a simple approach that leverages LLM embeddings for producing recommendations, can provide competitive performance by highlighting semantically related items. We publicly share the code and data of our experiments to ensure reproducibility.1

Despite their proven various benefits, Recommender Systems can cause or amplify certain undesired effects. In this paper, we focus on Popularity Bias, i.e., the tendency of a recommender system to utilize the effect of recommending popular items to the user. Prior research has studied the negative impact of this type of bias on individuals and society as a whole and proposed various approaches to mitigate this in various domains. However, almost all works adopted offline methodologies to evaluate the effectiveness of the proposed approaches. Unfortunately, such offline simulations can potentially be rather simplified and unable to capture the full picture. To contribute to this line of research and given a particular lack of knowledge about how debiasing approaches work not only offline, but online as well, we present in this paper the results of user study on a national broadcaster movie streaming platform in Norway, i.e., TV 2, following the A/B testing methodology. We deployed an effective mitigation approach for popularity bias, called Calibrated Popularity (CP), and monitored its performance in comparison to the platform’s existing collaborative filtering recommendation approach as a baseline over a period of almost four months. The results obtained from a large user base interacting in real-time with the recommendations indicate that the evaluated debiasing approach can be effective in addressing popularity bias while still maintaining the level of user interest and engagement.

The performance of a recommender system algorithm in terms of common offline accuracy measures often strongly depends on the chosen hyperparameters. Therefore, when comparing algorithms in offline experiments, we can obtain reliable insights regarding the effectiveness of a newly proposed algorithm only if we compare it to a number of state-of-the-art baselines that are carefully tuned for each of the considered datasets. While this fundamental principle of any area of applied machine learning is undisputed, we find that the tuning process for the baselines in the current literature is barely documented in much of today’s published research. Ultimately, in case the baselines are actually not carefully tuned, progress may remain unclear. In this paper, we exemplify through a computational experiment involving seven recent deep learning models how every method in such an unsound comparison can be reported to be outperforming the state-of-the-art. Finally, we iterate appropriate research practices to avoid unreliable algorithm comparisons in the future.

Reinforcement learning-based recommender systems have recently gained popularity. However, the design of the reward function, on which the agent relies to optimize its recommendation policy, is often not straightforward. Exploring the causality underlying users' behavior can take the place of the reward function in guiding the agent to capture the dynamic interests of users. Moreover, due to the typical limitations of simulation environments (e.g., data ineffi- ciency), most of the work cannot be broadly applied in large-scale situations. Although some works attempt to convert the offline dataset into a simulator, data inefficiency makes the learning pro- cess even slower. Because of the nature of reinforcement learning (i.e., learning by interaction), it cannot collect enough data to train during a single interaction. Furthermore, traditional reinforcement learning algorithms do not have a solid capability like supervised learning methods to learn from offline datasets directly. In this paper, we propose a new model named the causal decision transformer for recommender systems (CDT4Rec). CDT4Rec is an offline reinforce- ment learning system that can learn from a dataset rather than from online interaction. Moreover, CDT4Rec employs the transformer architecture, which is capable of processing large offline datasets and capturing both short-term and long-term dependencies within the data to estimate the causal relationship between action, state, and reward. To demonstrate the feasibility and superiority of our model, we have conducted experiments on six real-world offline datasets and one online simulator.

Complementary item recommendations are a ubiquitous feature of modern e-commerce sites. Such recommendations are highly effective when they are based on collaborative signals like co-purchase statistics. In certain online marketplaces, however, e.g., on online auction sites, constantly new items are added to the catalog. In such cases, complementary item recommendations are often based on item side-information due to a lack of interaction data. In this work, we propose a novel approach that can leverage both item side-information and labeled complementary item pairs to generate effective complementary recommendations for cold items, i.e., for items for which no co-purchase statistics yet exist. Given that complementary items typically have to be of a different category than the seed item, we technically maintain a latent space for each item category. Simultaneously, we learn to project distributed item representations into these category spaces to determine suitable recommendations. The main learning process in our architecture utilizes labeled pairs of complementary items. In addition, we adopt ideas from Cycle Generative Adversarial Networks (CycleGAN) to leverage available item information even in case no labeled data exists for a given item and category. Experiments on three e-commerce datasets show that our method is highly effective.

Recommender systems have become a ubiquitous part of our digital lives.Every day we receive suggestions about what to shop online, which video series to watch, what news to read, or which people to follow on social media.With their suggestions, these systems help us find relevant things when there is just too much information available or when there are too many alternatives to choose from.Over the past 25 years, recommender systems have become a research field of their own, transcending early roots in information retrieval and other areas of computer science.Indeed, building a successful recommender system in practice requires the cooperation of people with different backgrounds, from machine learning and data science experts to human-computer interaction and business professionals.Given the multi-faceted nature of recommender systems, research in this area has traditionally been scattered, with various publication outlets in different scientific subfields.This is why we launched ACM Transactions on Recommender Systems (ACM TORS).With this new journal, our goal is to be a home and preferred outlet for the latest high-quality research on the various aspects of recommender systems, from algorithms to the user experience, to questions of the impact and value of such systems.The journal welcomes submissions of various types, from original research works, to survey papers, reproducibility studies, industry reports, and opinion pieces.This inaugural issue contains a selection of articles on the latest research in our area.Several additional articles have already been accepted for future issues, and we are happy to observe that our research area is flourishing and continuously growing.The idea of a journal entirely devoted to recommender systems has been around for several years.Making this idea a reality would not have been possible without the support of many people.First, there was tremendous support from our research community, both from academia and industry, and from the Steering Committee of the ACM Conference on Recommender Systems.We are thankful that with this support ACM TORS can rely on an extraordinary board of Associate Editors, involving many top-notch international experts who have shaped the entire field over the years.Moreover, many very senior colleagues immediately volunteered to help as part of the board of Distinguished Reviewers, and researchers around the world have already served as reviewers for the articles we have received so far.Our specific thanks go to Joseph A. Konstan, who helped us in several important ways throughout the process, and to Joeran Beel, who is not only the Information Director of ACM TORS but also played an important role when we started our journey.We also thank the people at ACM and their supporting organizations for their help in ramping up this journal.This includes Laura Lander, Shayla Poling, and Starratt Scheetz, who were always there to deal with the questions and inquiries that have come up over the months.Finally, we thank you, the readers of ACM TORS.We are hoping that you will follow the subsequent issues of the journal and that you will enjoy the breadth and variety of the works that we will publish on the various facets of recommender systems research.We are open to your feedback to continuously improve the journal, and we hope that you may consider contributing to the journal in the future.

Automatically understanding the contents of an image is a highly relevant problem in practice. In e-commerce and social media settings, for example, a common problem is to automatically categorize user-provided pictures. Nowadays, a standard approach is to fine-tune pre-trained image models with application-specific data. Besides images, organizations however often also collect collaborative signals in the context of their application, in particular how users interacted with the provided online content, e.g., in forms of viewing, rating, or tagging. Such signals are commonly used for item recommendation, typically by deriving latent user and item representations from the data. In this work, we show that such collaborative information can be leveraged to improve the classification process of new images. Specifically, we propose a multitask learning framework, where the auxiliary task is to reconstruct collaborative latent item representations. A series of experiments on datasets from e-commerce and social media demonstrates that considering collaborative signals helps to significantly improve the performance of the main task of image classification by up to 9.1%.

Frequent updates and model retraining are important in various application areas of recommender systems, e.g., news recommendation. Moreover, in such domains, we may not only face the problem of dealing with a constant stream of new data, but also with anonymous users, leading to the problem of streaming session-based recommendation (SSR). Such problem settings have attracted increased interest in recent years, and different deep learning architectures were proposed that support fast updates of the underlying prediction models when new data arrive. In a recent paper, a method based on Graph Neural Networks (GNN) was proposed as being superior than previous methods for the SSR problem. The baselines in the reported experiments included different machine learning models. However, several earlier studies have shown that often conceptually simpler methods, e.g., based on nearest neighbors, can be highly effective for session-based recommendation problems. In this work, we report a similar phenomenon for the streaming configuration. We first reproduce the results of the mentioned GNN method and then show that simpler methods are able to outperform this complex state-of-the-art neural method on two datasets. Overall, our work points to continued methodological issues in the academic community, e.g., in terms of the choice of baselines and reproducibility.1

Conversational recommender systems (CRS) have attracted immense attention in the past few years. The most recent approaches rely on neural models trained on recorded dialogs between humans, implementing an end-to-end learning process. These systems are commonly designed to generate responses given the user’s utterances in natural language. One main challenge is that these generated responses both have to be appropriate for the given dialog context and must be grammatically and semantically correct. Recent studies however highlighted that current generation-based systems often struggle to provide such responses. An alternative to generation-based approaches is to retrieve responses from pre-recorded dialog data and to adapt them if needed. Such retrieval-based approaches were successfully explored in the context of general conversational systems, but have received limited attention in recent years for CRS. In this work, we re-assess the potential of such approaches and design and evaluate a novel technique for response retrieval and ranking. A user study (N = 90) revealed that the responses by our system were on average of higher quality than those of two recent generation-based systems. We furthermore found that the quality ranking of the two generation-based approaches is not aligned with the results from the literature, which points to open methodological questions. Overall, our research underlines that retrieval-based approaches should be considered an alternative or complement to language generation approaches.1

Sequential recommendation problems have received increased research interest in recent years. In such scenarios, the task is to suggest items to users to consume next, given their past interaction history, e.g., the next movie to watch or the next item to place in the shopping cart. A number of machine learning models were proposed recently for the task of sequential recommendation, with the latest ones based on deep learning techniques, in particular on Transformers. Given the often surprisingly competitive performance of simpler nearest-neighbor methods for the related problem of session-based recommendation, we investigate the use of nearest-neighbor methods for sequential recommendation problems. Our analysis on four datasets shows that nearest-neighbor methods achieve comparable or better performance than the recent Transformer-based bert4rec method on two of them. However, the deep learning method outperforms the simple methods for the two larger datasets, confirming previous hypotheses that neural methods work best when more data is available. As a further result of our experiments, we found additional evidence that sampled metrics must be used with care, as they may not be predictive of an algorithm ranking that would be observed with the non-sampled, full evaluation.

In the past few years we observed a renewed interest in conversational recommender systems (CRS) that interact with users in natural language. Most recent research efforts use neural models trained on recorded recommendation dialogs between humans, supporting an end-to-end learning process. Given the user’s utterances in a dialog, these systems aim to generate appropriate responses in natural language based on the learned models. An alternative to such language generation approaches is to retrieve and possibly adapt suitable sentences from the recorded dialogs. Approaches of this latter type are explored only to a lesser extent in the current literature.

Due to the advances in deep learning, visually-aware recommender systems (RS) have recently attracted increased research interest. Such systems combine collaborative signals with images, usually represented as feature vectors outputted by pre-trained image models. Since item catalogs can be huge, recommendation service providers often rely on images that are supplied by the item providers. In this work, we show that relying on such external sources can make an RS vulnerable to attacks, where the goal of the attacker is to unfairly promote certain pushed items. Specifically, we demonstrate how a new visual attack model can effectively influence the item scores and rankings in a black-box approach, i.e., without knowing the parameters of the model. The main underlying idea is to systematically create small human-imperceptible perturbations of the pushed item image and to devise appropriate gradient approximation methods to incrementally raise the pushed item's score. Experimental evaluations on two datasets show that the novel attack model is effective even when the contribution of the visual features to the overall performance of the recommender system is modest.

Recommender systems are designed to help users in situations of information overload. In recent years we observed increased interest in session-based recommendation scenarios, where the problem is to make item suggestions to users based only on interactions observed in an ongoing session, e.g., on an e-commerce site. However, in cases where interactions from previous user sessions are also available, the recommendations can be personalized according to the users’ long-term preferences, a process called session-aware recommendation. Today, research in this area is scattered, and many works only compare a newly proposed session-aware with existing session-based models. This makes it challenging to understand what represents the state-of-the-art. To close this research gap, we benchmarked recent session-aware algorithms against each other and against a number of session-based recommendation algorithms along with heuristic extensions thereof. Our comparison, to some surprise, revealed that (i) simple techniques based on nearest neighbors consistently outperform recent neural techniques and that (ii) session-aware models were mostly not better than approaches that do not use long-term preference information. Our work therefore points to potential methodological issues where new methods are compared to weak baselines, and it also indicates that there remains a huge potential for more sophisticated session-aware recommendation algorithms.

The prediction of the Customer Lifetime Value (CLV) is an important asset for tool-supported marketing by customer relationship managers. Since standard methods based on purchase recency, frequency, and past profit and revenue statistics often have limited predictive power, advanced machine learning (ML) techniques were applied to this task in recent years. However, existing approaches are often not fully capable of modeling certain temporal patterns that can be commonly found in practice, such as periodic purchasing behavior of customers. To address these shortcomings, we propose a novel method for CLV prediction based on a combination of several ML techniques. At its core, our method consists of a tailored deep learning approach based on encoder–decoder sequence-to-sequence recurrent neural networks with augmented temporal convolutions. This model is then combined with gradient boosting machines (GBMs) and a set of novel features in a hybrid framework. Empirical evaluations based on real-world data from a larger e-commerce company and a public dataset from the domain of online retail show that already the sequence-based model leads to competitive performance results. Stacking it with the GBM model is synergistic and further improves accuracy, indicating that the two models capture different patterns in the data.

The design of algorithms that generate personalized ranked item lists is a central topic of research in the field of recommender systems. In the past few years, in particular, approaches based on deep learning (neural) techniques have become dominant in the literature. For all of them, substantial progress over the state-of-the-art is claimed. However, indications exist of certain problems in today’s research practice, e.g., with respect to the choice and optimization of the baselines used for comparison, raising questions about the published claims. To obtain a better understanding of the actual progress, we have compared recent results in the area of neural recommendation approaches based on collaborative filtering against a consistent set of existing simple baselines. The worrying outcome of the analysis of these recent works—all were published at prestigious scientific conferences between 2015 and 2018—is that 11 of the 12 reproducible neural approaches can be outperformed by conceptually simple methods, e.g., based on the nearest-neighbor heuristic or linear models. None of the computationally complex neural methods was actually consistently better than already existing learning-based techniques, e.g., using matrix factorization or linear models. In our analysis, we discuss common issues in today’s research practice, which, despite the many papers that are published on the topic, have apparently led the field to a certain level of stagnation. 1

In recent years, algorithm research in the area of recommender systems has shifted from matrix factorization techniques and their latent factor models to neural approaches. However, given the proven power of latent factor models, some newer neural approaches incorporate them within more complex network architectures. One specific idea, recently put forward by several researchers, is to consider potential correlations between the latent factors, i.e., embeddings, by applying convolutions over the user-item interaction map. However, contrary to what is claimed in these articles, such interaction maps do not share the properties of images where Convolutional Neural Networks (CNNs) are particularly useful. In this work, we show through analytical considerations and empirical evaluations that the claimed gains reported in the literature cannot be attributed to the ability of CNNs to model embedding correlations, as argued in the original papers. Moreover, additional performance evaluations show that all of the examined recent CNN-based models are outperformed by existing non-neural machine learning techniques or traditional nearest-neighbor approaches. On a more general level, our work points to major methodological issues in recommender systems research.

Session-based recommendation is a problem setting where the task of a recommender system is to make suitable item suggestions based only on a few observed user interactions in an ongoing session. The lack of long-term preference information about individual users in such settings usually results in a limited level of personalization, where a small set of popular items may be recommended to many users. This repeated exposure of such a subset of the items through the recommendations may in turn lead to a reinforcement effect over time, and to a system which is not able to help users discover new content anymore to the desirable extent.

Recommender systems research is largely focused on the value such systems can create for users, e.g., by helping them finding items of interest in situations of information overload. However, there are various other ways in which recommender systems can create value and have an impact on individuals and organizations. The goal of the workshop is to serve as a platform where researchers discuss recent insights on how recommender systems affect individuals, user communities, or organizations. The workshop also aims at raising awareness regarding the importance of impact-oriented research.

eCommerce Information Retrieval (IR) is receiving increasing attention in the academic literature and is an essential component of some of the largest web sites (e.g. Amazon, Alibaba, Taobao, eBay, Airbnb, Target, Facebook). eCommerce organisations consistently sponsor SIGIR, reflecting the importance of IR research to them. This workshop (1) brings together researchers and practitioners of eCommerce IR to discuss topics unique to it, (2) determines how to use eCommerce's unique combination of free text, structured data, and customer behavioral data to improve search relevance, and (3) examines how to build data sets and evaluate algorithms in this domain. Since eCommerce customers often do not know exactly what they want to buy, recommendations are valuable for inspiration, serendipitous discovery and basket building. The theme of this year's eCommerce IR workshop is integrating recommendations into search for eCommerce. In addition to the focus on recommender systems in eCommerce search, Rakuten France is sponsoring a data challenge on taxonomy classification using multi-modal (image, text and structured data) input. The data challenge reflects themes from the 2017--2019 SIGIR workshops.

Deep learning techniques have become the method of choice for researchers working on algorithmic aspects of recommender systems. With the strongly increased interest in machine learning in general, it has, as a result, become difficult to keep track of what represents the state-of-the-art at the moment, e.g., for top-n recommendation tasks. At the same time, several recent publications point out problems in today's research practice in applied machine learning, e.g., in terms of the reproducibility of the results or the choice of the baselines when proposing new models.

The creation of an automated and virtually endless playlist given a start item is a common feature of modern media streaming services. When no past information about the user's preferences is available, the creation of such playlists can be done using session-based recommendation techniques. In this case, the recommendations only depend on the start item and the user's interactions in the current listening session, such as "liking" or skipping an item.

The benefits of neural approaches are undisputed in many application areas. However, today's research practice in applied machine learning---where researchers often use a variety of baselines, datasets, and evaluation procedures---can make it difficult to understand how much progress is actually achieved through novel technical approaches. In this work, we focus on the fast-developing area of session-based recommendation and aim to contribute to a better understanding of what represents the state-of-the-art.

Research in the area of recommender systems is largely focused on the value such a system creates for the users, by helping them finding items they are interested in. This is usually done by learning to rank the recommendable items based on their assumed relevance for each user. The implicit underlying goal often is that this personalization positively affects users in different positive ways, e.g., by making their search and decision processes easier or by helping them discover new things [3].

News is one of the earliest application domains of recommender systems, and recommending items from a virtually endless stream of news is still a relevant problem today. News recommendation is different from other application domains in a variety of ways, e.g., because new items constantly become available for recommendation. To be effective, news recommenders therefore have to continuously consider the latest items in the incoming stream of news in their recommendation models. However, today's public software libraries for algorithm benchmarking mostly do not consider these particularities of the domain. As a result, authors often rely on proprietary protocols, which hampers the comparability of the obtained results. In this paper, we present StreamingRec as a framework for evaluating streaming-based news recommenders in a replicable way. The open-source framework implements a replay-based evaluation protocol that allows algorithms to update the underlying models in real-time when new events are recorded and new articles are available for recommendation. Furthermore, a variety of baseline algorithms for session-based recommendation are part of StreamingRec. For these, we also report a number of performance results for two datasets, which confirm the importance of immediate model updates.

Deep learning methods have led to substantial progress in various application fields of AI, and in recent years a number of proposals were made to improve recommender systems with artificial neural networks. For the problem of making session-based recommendations, i.e., for recommending the next item in an anonymous session, Hidasi et al.~recently investigated the application of recurrent neural networks with Gated Recurrent Units (GRU4REC). Assessing the true effectiveness of such novel approaches based only on what is reported in the literature is however difficult when no standard evaluation protocols are applied and when the strength of the baselines used in the performance comparison is not clear. In this work we show based on a comprehensive empirical evaluation that a heuristics-based nearest neighbor (kNN) scheme for sessions outperforms GRU4REC in the large majority of the tested configurations and datasets. Neighborhood sampling and efficient in-memory data structures ensure the scalability of the kNN method. The best results in the end were often achieved when we combine the kNN approach with GRU4REC, which shows that RNNs can leverage sequential signals in the data that cannot be detected by the co-occurrence-based kNN method.

The purpose of recommenders is often summarized as "help the users find relevant items", and the predominant operationalization of this goal has been to focus on the ability to numerically estimate the users' preferences for unseen items or to provide users with item lists ranked in accordance to the estimated preferences. This dominant, albeit narrow, view of the recommendation problem has been tremendously helpful in advancing research in different ways, e.g., through the establishment of standardized evaluation procedures and metrics. In reality, recommender systems can serve a variety of purposes from the point of view of both consumers and providers. Most of the purposes, however, are significantly underexplored, even though many of them are arguably more aligned with the real-world expectations for recommenders than our current predominant paradigm. Therefore, it is important to revisit our conceptualizations of the potential goals of recommenders and their operationalization as research problems. In this paper, we discuss a framework of recommendation goals and purposes and highlight possible future directions and challenges related to the operationalization of such alternative problem formulations.

An essential characteristic in many e-commerce settings is that website visitors can have very specific short-term shopping goals when they browse the site. Relying solely on long-term user models that are pre-trained on historical data can therefore be insufficient for a suitable next-basket recommendation. Simple "real-time" recommendation approaches based, e.g., on unpersonalized co-occurrence patterns, on the other hand do not fully exploit the available information about the user's long-term preference profile. In this work, we aim to explore and quantify the effectiveness of using and combining long-term models and short-term adaptation strategies. We conducted an empirical evaluation based on a novel evaluation design and two real-world datasets. The results indicate that maintaining short-term content-based and recency-based profiles of the visitors can lead to significant accuracy increases. At the same time, the experiments show that the choice of the algorithm for learning the long-term preferences is particularly important at the beginning of new shopping sessions.

Automated playlist generation is a special form of music recommendation and a common feature of digital music playing applications. A particular challenge of the task is that the recommended items should not only match the general listener's preference but should also be coherent with the most recently played tracks. In this work, we propose a novel algorithmic approach and optimization scheme to generate playlist continuations that address these requirements. In our approach, we first use collections of shared music playlists, music metadata, and user preferences to select suitable tracks with high accuracy. Next, we apply a generic re-ranking optimization scheme to generate playlist continuations that match the characteristics of the last played tracks. An empirical evaluation on three collections of shared playlists shows that the combination of different input signals helps to achieve high accuracy during track selection and that the re-ranking technique can both help to balance different quality optimization goals and to further increase accuracy.

RapidMiner is a software tool that allows users to define data mining processes based on a visual model and implements a variety of so-called "operators" for data extraction, manipulation, model learning and analysis. The large number of available operators can however make it challenging for the process designer to find the appropriate operators for the problem at hand. At the same time, some operators are only meaningful when combined with certain others.

The emergence of what is called the social web and the continuing stream of new applications and community-based platforms including Facebook, Twitter, LinkedIn and others had a substantial impact on recommender systems research and practice over the last years in different ways.

In many application domains of recommender systems, explicit rating information is sparse or non-existent. The preferences of the current user have therefore to be approximated by interpreting his or her behavior, i.e., the implicit user feedback. In the literature, a number of algorithm proposals have been made that rely solely on such implicit feedback, among them Bayesian Personalized Ranking (BPR).

No abstract available.

In recent years, different proposals have been made to exploit Social Web tagging information to build more effective recommender systems. The tagging data, for example, were used to identify similar users or were viewed as additional information about the recommendable items. Recent research has indicated that “attaching feelings to tags” is experienced by users as a valuable means to express which features of an item they particularly like or dislike. When following such an approach, users would therefore not only add tags to an item as in usual Web 2.0 applications, but also attach a preference ( affect ) to the tag itself, expressing, for example, whether or not they liked a certain actor in a given movie. In this work, we show how this additional preference data can be exploited by a recommender system to make more accurate predictions. In contrast to previous work, which also relied on so-called tag preferences to enhance the predictive accuracy of recommender systems, we argue that tag preferences should be considered in the context of an item. We therefore propose new schemes to infer and exploit context-specific tag preferences in the recommendation process. An evaluation on two different datasets reveals that our approach is capable of providing more accurate recommendations than previous tag-based recommender algorithms and recent tag-agnostic matrix factorization techniques.

Product configuration can be defined as the task of tailoring a product according to the specific needs of a customer. Due to the inherent complexity of this task, which for example includes the consideration of complex constraints or the automatic completion of partial configurations, various Artificial Intelligence techniques have been explored in the last decades to tackle such configuration problems. Most of the existing approaches adopt a single-site, centralized approach. In modern supply chain settings, however, the components of a customizable product may themselves be configurable, thus requiring a multisite, distributed approach. In this paper, we analyze the challenges of modeling and solving such distributed configuration problems and propose an approach based on Distributed Constraint Satisfaction. In particular, we advocate the use of Generative Constraint Satisfaction for knowledge modeling and show in an experimental evaluation that the use of generic constraints is particularly advantageous also in the distributed problem solving phase.

No abstract available.

When we evaluate the quality of recommender systems (RS), most approaches only focus on the predictive accuracy of these systems. Recent works suggest that beyond accuracy there is a variety of other metrics that should be considered when evaluating a RS. In this paper we focus on two crucial metrics in RS evaluation: coverage and serendipity. Based on a literature review, we first discuss both measurement methods as well as the trade-off between good coverage and serendipity. We then analyze the role of coverage and serendipity as indicators of recommendation quality, present novel ways of how they can be measured and discuss how to interpret the obtained measurements. Overall, we argue that our new ways of measuring these concepts reflect the quality impression perceived by the user in a better way than previous metrics thus leading to enhanced user satisfaction.

Since the development of the comparably simple neighborhood-based methods in the 1990s, a plethora of techniques has been developed to improve various aspects of collaborative filtering recommender systems like predictive accuracy, scalability to large problem instances or the capability to deal with sparse data sets. Many of the recent algorithms rely on sophisticated methods which are based, for instance, on matrix factorization techniques or advanced probabilistic models and/or require a computationally intensive model-building phase. In this work, we evaluate the accuracy of a new and extremely simple prediction method (RF-Rec) that uses the user's and the item's most frequent rating value to make a rating prediction. The evaluation on three standard test data sets shows that the accuracy of the algorithm is on a par with the standard collaborative filtering algorithms on dense data sets and outperforms them on sparse rating databases. At the same time, the algorithm's implementation is trivial, has a high prediction coverage, requires no complex offline pre-processing or model-building phase and can generate predictions in constant time.

No abstract available.

This paper summarizes the initial findings of an experimental evaluation of how recommender systems affect the buying behavior of online customers. The study was conducted in the context of a large-scale, commercial Mobile Internet platform, from which end users can download games to their mobile phones. Item recommendations were presented to platform visitors in different navigational situations; the recommendation lists were either determined with the help of different recommendation algorithms or based on nonpersonalized ranking techniques. The study is based on a sample of more than 155,000 different customers who visited the portal during a four week evaluation period. The analysis revealed that the use of personalized recommendations instead of non-personalized ones leads to a significant increase in viewed and sold items in different navigational situations and to an overall sales increase.

Web mining systems exploit the redundancy of data published on the Web to automatically extract information from existing Web documents. The first step in the information extraction process is thus to locate within a limited period of time as many Web pages as possible that contain relevant information, a task which is commonly accomplished by applying focused crawling techniques. The performance of such a crawler can be measured by its "recall", i.e. the percentage of documents found and identified as relevant compared to the number of existing documents. A higher recall value implies that more redundant data is available, which in turn leads to better results in the subsequent fact extraction phase. In this paper, we propose xCrawl, a new focused crawling method which outperforms state-of-the-art approaches with respect to recall values achievable within a given period of time. This method is based on a new combination of ideas and techniques used to identify and exploit navigational structures of Websites, such as hierarchies, lists or maps. In addition, automatic query generation is applied to rapidly collect Web sources containing target documents. The proposed crawling technique was inspired by the requirements of a Web mining system developed to extract product and service descriptions and was evaluated in different application scenarios. Comparisons with existing focused crawling techniques reveal that the new crawling method leads to a significant increase in recall whilst maintaining precision.

One of the common approaches to extracting high-quality knowledge from Web sources is to exploit the redundancy of the published information. Therefore, a Web Mining System not only has to search for relevant Web pages but also has to somehow determine whether two pages describe the same entity in order to extract as much knowledge as possible about it. It has been shown that statistical clustering techniques are in general a suitable means to achieve this task by grouping documents that are supposed to contain similar information. However, when data is given in tabular form - which is for instance a typical way of describing items in online shops - existing document clustering algorithms show limited performance as documents containing tabular descriptions typically share a very common set of tokens although they describe different entities. In this paper we therefore propose a new document clustering approach that exploits hyperlinks and document metadata to extract candidates for entity names. These candidate names are subsequently used to cluster the documents and further improve these names, which are finally used to determine whether two documents describe the same entity. The detailed evaluation of our approach in two popular example domains showed its high accuracy in terms of precision and recall (F-Measure > 0.9).