Novel data sources bring new opportunities to improve the quality of recommender systems and serve as a catalyst for the creation of new paradigms on personalized recommendations. Impressions are a novel data source containing the items shown to users on their screens. Past research focused on providing personalized recommendations using interactions and occasionally using impressions when such a data source was available. Interest in impressions has increased due to their potential to provide more accurate recommendations. Despite this increased interest, research in recommender systems using impressions is still dispersed. Many works have distinct interpretations of impressions and use impressions in recommender systems in numerous different manners. To unify those interpretations into a single framework, we present a systematic literature review on recommender systems using impressions, focusing on three fundamental perspectives: recommendation models , datasets , and evaluation methodologies . We define a theoretical framework to delimit recommender systems using impressions and a novel paradigm for personalized recommendations, called impression-aware recommender systems. We propose a classification system for recommenders in this paradigm, which we use to categorize the recommendation models, datasets, and evaluation methodologies used in past research. Last, we identify open questions and future directions, highlighting missing aspects in the reviewed literature.

The field of Quantum Computing (QC) has gained significant popularity in recent years, due to its potential to provide benefits in terms of efficiency and effectiveness when employed to solve certain computationally intensive tasks. In both Information Retrieval (IR) and Recommender Systems (RS) we are required to build methods that apply complex processing on large and heterogeneous datasets, it is natural therefore to wonder whether QC could also be applied to boost their performance. The tutorial aims to provide first an introduction to QC for an audience that is not familiar with the technology, then to show how to apply the QC paradigm of Quantum Annealing (QA) to solve practical problems that are currently faced by IR and RS systems. During the tutorial, participants will be provided with the fundamentals required to understand QC and to apply it in practice by using a real D-Wave quantum annealer through APIs.

Recommender systems typically rely on past user interactions as the primary source of information for making predictions. However, although highly informative, past user interactions are strongly biased. Impressions, on the other hand, are a new source of information that indicate the items displayed on screen when the user interacted (or not) with them, and have the potential to impact the field of recommender systems in several ways. Early research on impressions was constrained by the limited availability of public datasets, but this is rapidly changing and, as a consequence, interest in impressions has increased. Impressions present new research questions and opportunities, but also bring new challenges. Several works propose to use impressions as part of recommender models in various ways and discuss their information content. Others explore their potential in off-policy-estimation and reinforcement learning. Overall, the interest of the community is growing, but efforts in this direction remain disconnected. Therefore, we believe that a workshop would be useful in bringing the community together.

In Recommender Systems, impressions are a relatively new type of information that records all products previously shown to the users. They are also a complex source of information, combining the effects of the recommender system that generated them, search results, or business rules that may select specific products for recommendations. The fact that the user interacted with a specific item given a list of recommended ones may benefit from a richer interaction signal, in which some items the user did not interact with may be considered negative interactions. This work presents a preliminary evaluation of recommendation models with impressions. First, impressions are characterized by describing their assumptions, signals, and challenges. Then, an evaluation study with impressions is described. The study’s goal is two-fold: to measure the effects of impressions data on properly-tuned recommendation models using current open-source datasets and disentangle the signals within impressions data. Preliminary results suggest that impressions data and signals are nuanced, complex, and effective at improving the recommendation quality of recommenders. This work publishes the source code, datasets, and scripts used in the evaluation to promote reproducibility in the domain.

After decades of being mainly confined to theoretical research, Quantum Computing is now becoming a useful tool for solving realistic problems. This work aims to experimentally explore the feasibility of using currently available quantum computers, based on the Quantum Annealing paradigm, to build a recommender system exploiting community detection. Community detection, by partitioning users and items into densely connected clusters, can boost the accuracy of non-personalized recommendation by assuming that users within each community share similar tastes. However, community detection is a computationally expensive process. The recent availability of Quantum Annealers as cloud-based devices, constitutes a new and promising direction to explore community detection, although effectively leveraging this new technology is a long-term path that still requires advancements in both hardware and algorithms. This work aims to begin this path by assessing the quality of community detection formulated as a Quadratic Unconstrained Binary Optimization problem on a real recommendation scenario. Results on several datasets show that the quantum solver is able to detect communities of comparable quality with respect to classical solvers, but with better speedup, and the non-personalized recommendation models built on top of these communities exhibit improved recommendation quality. The takeaway is that quantum computing, although in its early stages of maturity and applicability, shows promise in its ability to support new recommendation models and to bring improved scalability as technology evolves.

Feature selection is a common step in many ranking, classification, or prediction tasks and serves many purposes. By removing redundant or noisy features, the accuracy of ranking or classification can be improved and the computational cost of the subsequent learning steps can be reduced. However, feature selection can be itself a computationally expensive process. While for decades confined to theoretical algorithmic papers, quantum computing is now becoming a viable tool to tackle realistic problems, in particular special-purpose solvers based on the Quantum Annealing paradigm. This paper aims to explore the feasibility of using currently available quantum computing architectures to solve some quadratic feature selection algorithms for both ranking and classification.

Adding confidence estimates to predicted ratings has been shown to positively influence the quality of the recommendations provided by a recommender system. While confidence over single point predictions of ratings and preferences has been widely studied in literature, limited effort has been put in exploring the benefits provided by user-level confidence indices. In this work we exploit a recently introduced user-level confidence index, called eigenvalue confidence index, in order to provide maximum confidence recommendations for item-based recommender systems. We firstly derive a closed form solution to calculate the index, then we propose a new recommendation methodology for item-based models, called eigenvalue perturbation, founded on the strongly positive correlation between the index value and the accuracy of the recommendations. We show and discuss the accuracy results obtained with a comprehensive set of experiments over several datasets and using different item-based models, empirically proving that applying the new technique we are able to outperform the original recommendation models in most of the experimental configurations.

It has been long known that quantum computing has the potential to revolutionize the way we find solutions of problems that are difficult to solve on classical computers. It was only recently that small but functional quantum computers have become available on the cloud, allowing to test their potential. In this paper we propose to leverage their capabilities to address an important task for recommender systems providers, the optimal selection of recommendation carousels. In many video-on-demand and music streaming services the user is provided with a homepage containing several recommendation lists, i.e., carousels, each built with a certain criteria (e.g., artist, mood, Action movies etc.). Choosing which set of carousels to display is a difficult problem because it needs to account for how the different recommendation lists interact, e.g., avoiding duplicate recommendations, and how they help the user explore the catalogue. We focus in particular on the adiabatic computing paradigm and use the D-Wave quantum annealer, which is able to solve NP-hard optimization problems, can be programmed by classical operations research tools and is freely available on the cloud. We propose a formulation of the carousel selection problem for black box recommenders, that can be solved effectively on a quantum annealer and has the advantage of being simple. We discuss its effectiveness, limitations and possible future directions of development.

Properly selecting the configuration of a database management system (DBMS) is essential to increase performance and reduce costs. However, the task is astonishingly tricky due to a large number of tunable configuration parameters and their inter-dependencies. Also, the optimal configuration depends upon the workload to which the DBMS is exposed. To extract the full potential of a DBMS, we must also consider the entire IT stack on which the DBMS is running, comprising layers like the Java virtual machine, the operating system and the physical machine. Each layer offers a multitude of parameters that we should take into account. The available parameters vary as new software versions are released, making it impractical to rely on historical knowledge bases. We present a novel tuning approach for the DBMS configuration auto-tuning that quickly finds a well-performing configuration of an IT stack and adapts it to workload variations, without having to rely on a knowledge base. We evaluate the proposed approach using the Cassandra and MongoDB DBMSs, showing that it adjusts the suggested configuration to the observed workload and is portable across different IT applications. We try to minimise the memory consumption without increasing the response time, showing that the proposed approach reduces the response time and increases the memory requirements only under heavy-load conditions, reducing it again when the load decreases.

The design of algorithms that generate personalized ranked item lists is a central topic of research in the field of recommender systems. In the past few years, in particular, approaches based on deep learning (neural) techniques have become dominant in the literature. For all of them, substantial progress over the state-of-the-art is claimed. However, indications exist of certain problems in today’s research practice, e.g., with respect to the choice and optimization of the baselines used for comparison, raising questions about the published claims. To obtain a better understanding of the actual progress, we have compared recent results in the area of neural recommendation approaches based on collaborative filtering against a consistent set of existing simple baselines. The worrying outcome of the analysis of these recent works—all were published at prestigious scientific conferences between 2015 and 2018—is that 11 of the 12 reproducible neural approaches can be outperformed by conceptually simple methods, e.g., based on the nearest-neighbor heuristic or linear models. None of the computationally complex neural methods was actually consistently better than already existing learning-based techniques, e.g., using matrix factorization or linear models. In our analysis, we discuss common issues in today’s research practice, which, despite the many papers that are published on the topic, have apparently led the field to a certain level of stagnation. 1

In recent years, algorithm research in the area of recommender systems has shifted from matrix factorization techniques and their latent factor models to neural approaches. However, given the proven power of latent factor models, some newer neural approaches incorporate them within more complex network architectures. One specific idea, recently put forward by several researchers, is to consider potential correlations between the latent factors, i.e., embeddings, by applying convolutions over the user-item interaction map. However, contrary to what is claimed in these articles, such interaction maps do not share the properties of images where Convolutional Neural Networks (CNNs) are particularly useful. In this work, we show through analytical considerations and empirical evaluations that the claimed gains reported in the literature cannot be attributed to the ability of CNNs to model embedding correlations, as argued in the original papers. Moreover, additional performance evaluations show that all of the examined recent CNN-based models are outperformed by existing non-neural machine learning techniques or traditional nearest-neighbor approaches. On a more general level, our work points to major methodological issues in recommender systems research.

In this article, we introduce the \dataset dataset, a collection of implicit interactions and impressions of movies and TV series from an Over-The-Top media service, which delivers its media contents over the Internet. The dataset is distinguished from other already available multimedia recommendation datasets by the availability of impressions, \idest the recommendations shown to the user, its size, and by being open-source. We describe the data collection process, the preprocessing applied, its characteristics, and statistics when compared to other commonly used datasets. We also highlight several possible use cases and research questions that can benefit from the availability of user impressions in an open-source dataset. Furthermore, we release software tools to load and split the data, as well as examples of how to use both user interactions and impressions in several common recommendation algorithms.

Deep learning techniques have become the method of choice for researchers working on algorithmic aspects of recommender systems. With the strongly increased interest in machine learning in general, it has, as a result, become difficult to keep track of what represents the state-of-the-art at the moment, e.g., for top-n recommendation tasks. At the same time, several recent publications point out problems in today's research practice in applied machine learning, e.g., in terms of the reproducibility of the results or the choice of the baselines when proposing new models.

We propose a multi-modal content-based movie recommender system that replaces human-generated metadata with content descriptions automatically extracted from the visual and audio channels of a video. Content descriptors improve over traditional metadata in terms of both richness (it is possible to extract hundreds of meaningful features covering various modalities) and quality (content features are consistent across different systems and immune to human errors). Our recommender system integrates state-of-the-art aesthetic and deep visual features as well as block-level and i-vector audio features. For fusing the different modalities, we propose a rank aggregation strategy extending the Borda count approach.

In recent years, more and more recommendation algorithms have been proposed that are based on time-ordered user interaction logs. Algorithms for session-based recommendation tasks are among the most prominent examples of such approaches.

In the last years, there has been much attention given to the semantic gap problem in multimedia retrieval systems. Much effort has been devoted to bridge this gap by building tools for the extraction of high-level, semantics-based features from multimedia content, as low-level features are not considered useful because they deal primarily with representing the perceived content rather than the semantics of it.

Session-based recommendations are highly relevant in many modern on-line services (e.g. e-commerce, video streaming) and recommendation settings. Recently, Recurrent Neural Networks have been shown to perform very well in session-based settings. While in many session-based recommendation domains user identifiers are hard to come by, there are also domains in which user profiles are readily available. We propose a seamless way to personalize RNN models with cross-session information transfer and devise a Hierarchical RNN model that relays end evolves latent hidden states of the RNNs across user sessions. Results on two industry datasets show large improvements over the session-only RNNs.

In this position paper, we take the experimental approach of putting algorithms aside, and reflect on what recommenders would be for people if they were not tied to technology. By looking at some of the shortcomings that current recommenders have fallen into and discussing their limitations from a human point of view, we ask the question: if freed from all limitations, what should, and what could, RecSys be? We then turn to the idea that life itself is the best recommender system, and that people themselves are the query. By looking at how life brings people in contact with options that suit their needs or match their preferences, we hope to shed further light on what current RecSys could be doing better. Finally, we look at the forms that RecSys could take in the future. By formulating our vision beyond the reach of usual considerations and current limitations, including business models, algorithms, data sets, and evaluation methodologies, we attempt to arrive at fresh conclusions that may inspire the next steps taken by the community of researchers working on RecSys.

A critical change has occurred in the status of context in recommender systems. In the past, context has been considered 'additional evidence'. This past picture is at odds with many present application domains, where user and item information is scarce. Such domains face continuous cold start conditions and must exploit session rather than user information. In this paper, we describe the `Contextual Turn?: the move towards context-driven recommendation algorithms for which context is critical, rather than additional. We cover application domains, algorithms that promise to address the challenges of context-driven recommendation, and the steps that the community has taken to tackle context-driven problems. Our goal is to point out the commonalities of context-driven problems, and urge the community to address the overarching challenges that context-driven recommendation poses.

other Share on Introduction to the Special Issue on Recommender System Benchmarking Authors: Paolo Cremonesi Politecnico di Milano Politecnico di MilanoView Profile , Alan Said Recorded Future Recorded FutureView Profile , Domonkos Tikk Gravity R&D, Hungary Gravity R&D, HungaryView Profile , Michelle X. Zhou Juji JujiView Profile Authors Info & Claims ACM Transactions on Intelligent Systems and TechnologyVolume 7Issue 3April 2016 Article No.: 38pp 1–4https://doi.org/10.1145/2870627Published:08 March 2016Publication History 2citation384DownloadsMetricsTotal Citations2Total Downloads384Last 12 Months12Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Evaluation is a cardinal issue in recommender systems; as in any technical discipline, it highlights to a large extent the problems that need to be solved by the field and, hence, leads the way for algorithmic research and development in the community. Yet, in the field of recommender systems, there still exists considerable disparity in evaluation methods, metrics and experimental designs, as well as a significant mismatch between evaluation methods in the lab and what constitutes an effective recommendation for real users and businesses. Even after the relevant quality dimensions have been defined, a clear evaluation protocol should be specified in detail and agreed upon, allowing for the comparison of results and experiments conducted by different authors. This would enable any contribution to the same problem to be incremental and add up on top of previous work, rather than grow sideways. The REDD 2014 workshop seeks to provide an informal forum to tackle such issues and to move towards better understood and shared evaluation methodologies, allowing one to leverage the efforts and the workforce of the academic community towards meaningful and relevant directions in real-world developments.

Cross-domain recommender systems aim to generate or enhance personalized recommendations in a target domain by exploiting knowledge (mainly user preferences) from other source domains. This may beneficial for generating better recommendations, e.g. mitigating the cold-start and sparsity problems in a target domain, and enabling personalized cross-selling for items from multiple domains. In this tutorial, we formalize the cross-domain recommendation problem, categorize and survey state of the art cross-domain recommender systems, discuss related evaluation issues, and outline future research directions on the topic.

Cross-domain recommender systems adopt different techniques to transfer learning from source domain to target domain in order to alleviate the sparsity problem and improve accuracy of recommendations. Traditional techniques require the two domains to be linked by shared characteristics associated to either users or items. In collaborative filtering (CF) this happens when the two domains have overlapping users or item (at least partially). Recently, Li et al. [7] introduced codebook transfer (CBT), a cross-domain CF technique based on co-clustering, and presented experimental results showing that CBT is able to transfer knowledge between non-overlapping domains. In this paper, we disprove these results and show that CBT does not transfer knowledge when source and target domains do not overlap.

The CrowdRec workshop brings together the recommender system community for discussion and exchange of ideas. Its goal is to allow the potential of human computation and crowdsourcing to be exploited fully and sustainably, leading to the development of improved recommendation and information filtering technologies. Currently, the complete range of possible intelligent contributions that recommender systems could elicit from users is under-explored, and its full extent is unknown. Critical questions addressed in the workshop include how to: formulate crowdtasks, match tasks with crowdmembers, ensure the quality of crowd input, and integrate feedback from the crowd in an optimal manner to improve recommendation. Further, crowdsourcing can also be exploited for system design and system evaluation.

In a number of domains of interest for recommender systems, items are characterized by constrained and variable "capacity": the same product or service can be consumed by a limited number of users and the possibility of item consumption depends on contextual circumstances (e.g., time). Our work explores recommenders in the context of these "bounded" domains. We consider online hotel booking as a case study, and investigates if and how "missing" items (hotels that eventually becomes unavailable for users' consumption) affect the quality of recommendations. The paper proposes a technique for defining "missing" items as "best items", and presents an articulated empirical research in which recommendations for hotel online booking are evaluated in different experimental conditions with a user centric approach involving 142 participants.

One of the unresolved issues when designing a recommender system is the number of ratings -- i.e., the profile length -- that should be collected from a new user before providing recommendations. A design tension exists, induced by two conflicting requirements. On the one hand, the system must collect "enough"ratings from the user in order to learn her/his preferences and improve the accuracy of recommendations. On the other hand, gathering more ratings adds a burden on the user, which may negatively affect the user experience. Our research investigates the effects of profile length from both a subjective (user-centric) point of view and an objective (accuracy-based) perspective. We carried on an offline simulation with three algorithms, and a set of online experiments involving overall 960 users and four recommender algorithms, to measure which of the two contrasting forces influenced by the number of collected ratings -- recommendations relevance and burden of the rating process -- has stronger effects on the perceived quality of the user experience. Moreover, our study identifies the potentially optimal profile length for an explicit, rating based, and human controlled elicitation strategy.

In many commercial systems, the 'best bet' recommendations are shown, but the predicted rating values are not. This is usually referred to as a top-N recommendation task, where the goal of the recommender system is to find a few specific items which are supposed to be most appealing to the user. Common methodologies based on error metrics (such as RMSE) are not a natural fit for evaluating the top-N recommendation task. Rather, top-N performance can be directly measured by alternative methodologies based on accuracy metrics (such as precision/recall).

In this paper we evaluate the performance of different collaborative algorithms in cold-start situations, where the initial lack of ratings may affect the quality of the algorithms. The evaluation has been performed on the pay-per-view datasets collected by two IP-television providers over a period of several months. The analysis shows that item-based algorithms perform better with respect to SVD-based ones in the early stage of the cold-start problem. Moreover, the accuracy of SVD-based algorithms, when using few latent factors, decreases with the time-evolution of the dataset. On the contrary, the same algorithms used with a large-enough number of latent features increase their accuracy with time and may outperform the item-based algorithms.