Novel data sources bring new opportunities to improve the quality of recommender systems and serve as a catalyst for the creation of new paradigms on personalized recommendations. Impressions are a novel data source containing the items shown to users on their screens. Past research focused on providing personalized recommendations using interactions and occasionally using impressions when such a data source was available. Interest in impressions has increased due to their potential to provide more accurate recommendations. Despite this increased interest, research in recommender systems using impressions is still dispersed. Many works have distinct interpretations of impressions and use impressions in recommender systems in numerous different manners. To unify those interpretations into a single framework, we present a systematic literature review on recommender systems using impressions, focusing on three fundamental perspectives: recommendation models , datasets , and evaluation methodologies . We define a theoretical framework to delimit recommender systems using impressions and a novel paradigm for personalized recommendations, called impression-aware recommender systems. We propose a classification system for recommenders in this paradigm, which we use to categorize the recommendation models, datasets, and evaluation methodologies used in past research. Last, we identify open questions and future directions, highlighting missing aspects in the reviewed literature.

The field of Quantum Computing (QC) has gained significant popularity in recent years, due to its potential to provide benefits in terms of efficiency and effectiveness when employed to solve certain computationally intensive tasks. In both Information Retrieval (IR) and Recommender Systems (RS) we are required to build methods that apply complex processing on large and heterogeneous datasets, it is natural therefore to wonder whether QC could also be applied to boost their performance. The tutorial aims to provide first an introduction to QC for an audience that is not familiar with the technology, then to show how to apply the QC paradigm of Quantum Annealing (QA) to solve practical problems that are currently faced by IR and RS systems. During the tutorial, participants will be provided with the fundamentals required to understand QC and to apply it in practice by using a real D-Wave quantum annealer through APIs.

Recommender systems typically rely on past user interactions as the primary source of information for making predictions. However, although highly informative, past user interactions are strongly biased. Impressions, on the other hand, are a new source of information that indicate the items displayed on screen when the user interacted (or not) with them, and have the potential to impact the field of recommender systems in several ways. Early research on impressions was constrained by the limited availability of public datasets, but this is rapidly changing and, as a consequence, interest in impressions has increased. Impressions present new research questions and opportunities, but also bring new challenges. Several works propose to use impressions as part of recommender models in various ways and discuss their information content. Others explore their potential in off-policy-estimation and reinforcement learning. Overall, the interest of the community is growing, but efforts in this direction remain disconnected. Therefore, we believe that a workshop would be useful in bringing the community together.

The Off-Policy Evaluation (OPE) problem consists in evaluating the performance of new policies from the data collected by another one. OPE is crucial when evaluating a new policy online is too expensive or risky. Many of the state-of-the-art OPE estimators are based on the Inverse Propensity Scoring (IPS) technique, which provides an unbiased estimator when the full support assumption holds, i.e., when the logging policy assigns a non-zero probability to each action. However, there are several scenarios where this assumption does not hold in practice, i.e., there is deficient support, and the IPS estimator is biased in the general case.In this paper, we consider two alternative estimators for the deficient support OPE problem. We first show how to adapt an estimator that was originally proposed for a different domain to the deficient support setting.Then, we propose another estimator, which is a novel contribution of this paper.These estimators exploit additional information about the actions, which we call side information, in order to make reliable estimates on the unsupported actions. Under alternative assumptions that do not require full support, we show that the considered estimators are unbiased.We also provide a theoretical analysis of the concentration when relaxing all the assumptions. Finally, we provide an experimental evaluation showing how the considered estimators are better suited for the deficient support setting compared to the baselines.

In Recommender Systems, impressions are a relatively new type of information that records all products previously shown to the users. They are also a complex source of information, combining the effects of the recommender system that generated them, search results, or business rules that may select specific products for recommendations. The fact that the user interacted with a specific item given a list of recommended ones may benefit from a richer interaction signal, in which some items the user did not interact with may be considered negative interactions. This work presents a preliminary evaluation of recommendation models with impressions. First, impressions are characterized by describing their assumptions, signals, and challenges. Then, an evaluation study with impressions is described. The study’s goal is two-fold: to measure the effects of impressions data on properly-tuned recommendation models using current open-source datasets and disentangle the signals within impressions data. Preliminary results suggest that impressions data and signals are nuanced, complex, and effective at improving the recommendation quality of recommenders. This work publishes the source code, datasets, and scripts used in the evaluation to promote reproducibility in the domain.

After decades of being mainly confined to theoretical research, Quantum Computing is now becoming a useful tool for solving realistic problems. This work aims to experimentally explore the feasibility of using currently available quantum computers, based on the Quantum Annealing paradigm, to build a recommender system exploiting community detection. Community detection, by partitioning users and items into densely connected clusters, can boost the accuracy of non-personalized recommendation by assuming that users within each community share similar tastes. However, community detection is a computationally expensive process. The recent availability of Quantum Annealers as cloud-based devices, constitutes a new and promising direction to explore community detection, although effectively leveraging this new technology is a long-term path that still requires advancements in both hardware and algorithms. This work aims to begin this path by assessing the quality of community detection formulated as a Quadratic Unconstrained Binary Optimization problem on a real recommendation scenario. Results on several datasets show that the quantum solver is able to detect communities of comparable quality with respect to classical solvers, but with better speedup, and the non-personalized recommendation models built on top of these communities exhibit improved recommendation quality. The takeaway is that quantum computing, although in its early stages of maturity and applicability, shows promise in its ability to support new recommendation models and to bring improved scalability as technology evolves.

Feature selection is a common step in many ranking, classification, or prediction tasks and serves many purposes. By removing redundant or noisy features, the accuracy of ranking or classification can be improved and the computational cost of the subsequent learning steps can be reduced. However, feature selection can be itself a computationally expensive process. While for decades confined to theoretical algorithmic papers, quantum computing is now becoming a viable tool to tackle realistic problems, in particular special-purpose solvers based on the Quantum Annealing paradigm. This paper aims to explore the feasibility of using currently available quantum computing architectures to solve some quadratic feature selection algorithms for both ranking and classification.

Abstract New items, also called cold-start items, are introduced every day in the catalogs of numerous online systems. Due to the absence of previous preferences, recommending these items is difficult but important task for a recommender system. For this reason, the item cold-start recommendation problem still represents an interesting research topic for the community. In this work, we propose Neural Feature Combiner (NFC), a novel deep learning, item-based approach for cold-start item recommendation. The model learns to map the content features of the items into a low-dimensional hybrid embedding space. The features that compose the embeddings are then combined in order to reproduce collaborative item similarity values. We compare NFC with three variants of the same model that learn from user feedback, showing the advantages of learning from similarities in terms of accuracy and convergence time. With an extensive set of experiments on four datasets, we show that NFC outperforms several cutting edge approaches in the top-n recommendation task of cold-start items. Results in extremely cold (i.e., with a very low amount of interactions for training) and cold-warm hybrid scenarios prove that NFC effectively exploits collaborative information, leading to state-of-the-art accuracy. We finally conduct a qualitative analysis of the embeddings generated by different models, and we provide an analysis of the importance that different models assign to the input features, empirically demonstrating the robustness of the hybrid representations produced by our new model.

Adding confidence estimates to predicted ratings has been shown to positively influence the quality of the recommendations provided by a recommender system. While confidence over single point predictions of ratings and preferences has been widely studied in literature, limited effort has been put in exploring the benefits provided by user-level confidence indices. In this work we exploit a recently introduced user-level confidence index, called eigenvalue confidence index, in order to provide maximum confidence recommendations for item-based recommender systems. We firstly derive a closed form solution to calculate the index, then we propose a new recommendation methodology for item-based models, called eigenvalue perturbation, founded on the strongly positive correlation between the index value and the accuracy of the recommendations. We show and discuss the accuracy results obtained with a comprehensive set of experiments over several datasets and using different item-based models, empirically proving that applying the new technique we are able to outperform the original recommendation models in most of the experimental configurations.

It has been long known that quantum computing has the potential to revolutionize the way we find solutions of problems that are difficult to solve on classical computers. It was only recently that small but functional quantum computers have become available on the cloud, allowing to test their potential. In this paper we propose to leverage their capabilities to address an important task for recommender systems providers, the optimal selection of recommendation carousels. In many video-on-demand and music streaming services the user is provided with a homepage containing several recommendation lists, i.e., carousels, each built with a certain criteria (e.g., artist, mood, Action movies etc.). Choosing which set of carousels to display is a difficult problem because it needs to account for how the different recommendation lists interact, e.g., avoiding duplicate recommendations, and how they help the user explore the catalogue. We focus in particular on the adiabatic computing paradigm and use the D-Wave quantum annealer, which is able to solve NP-hard optimization problems, can be programmed by classical operations research tools and is freely available on the cloud. We propose a formulation of the carousel selection problem for black box recommenders, that can be solved effectively on a quantum annealer and has the advantage of being simple. We discuss its effectiveness, limitations and possible future directions of development.

It is common for video-on-demand and music streaming services to adopt a user interface composed of several recommendation lists, i.e., widgets or swipeable carousels, each generated according to a specific criterion or algorithm (e.g., most recent, top popular, recommended for you, editors’ choice, etc.). Selecting the appropriate combination of carousel has significant impact on user satisfaction. A crucial aspect of this user interface is that to measure the relevance a new carousel for the user it is not sufficient to account solely for its individual quality. Instead, it should be considered that other carousels will already be present in the interface. This is not considered by traditional evaluation protocols for recommenders systems, in which each carousel is evaluated in isolation, regardless of (i) which other carousels are displayed to the user and (ii) the relative position of the carousel with respect to other carousels. Hence, we propose a two-dimensional evaluation protocol for a carousel setting that will measure the quality of a recommendation carousel based on how much it improves upon the quality of an already available set of carousels. Our evaluation protocol takes into account also the position bias, i.e., users do not explore the carousels sequentially, but rather concentrate on the top-left corner of the screen.

Properly selecting the configuration of a database management system (DBMS) is essential to increase performance and reduce costs. However, the task is astonishingly tricky due to a large number of tunable configuration parameters and their inter-dependencies. Also, the optimal configuration depends upon the workload to which the DBMS is exposed. To extract the full potential of a DBMS, we must also consider the entire IT stack on which the DBMS is running, comprising layers like the Java virtual machine, the operating system and the physical machine. Each layer offers a multitude of parameters that we should take into account. The available parameters vary as new software versions are released, making it impractical to rely on historical knowledge bases. We present a novel tuning approach for the DBMS configuration auto-tuning that quickly finds a well-performing configuration of an IT stack and adapts it to workload variations, without having to rely on a knowledge base. We evaluate the proposed approach using the Cassandra and MongoDB DBMSs, showing that it adjusts the suggested configuration to the observed workload and is portable across different IT applications. We try to minimise the memory consumption without increasing the response time, showing that the proposed approach reduces the response time and increases the memory requirements only under heavy-load conditions, reducing it again when the load decreases.

The design of algorithms that generate personalized ranked item lists is a central topic of research in the field of recommender systems. In the past few years, in particular, approaches based on deep learning (neural) techniques have become dominant in the literature. For all of them, substantial progress over the state-of-the-art is claimed. However, indications exist of certain problems in today’s research practice, e.g., with respect to the choice and optimization of the baselines used for comparison, raising questions about the published claims. To obtain a better understanding of the actual progress, we have compared recent results in the area of neural recommendation approaches based on collaborative filtering against a consistent set of existing simple baselines. The worrying outcome of the analysis of these recent works—all were published at prestigious scientific conferences between 2015 and 2018—is that 11 of the 12 reproducible neural approaches can be outperformed by conceptually simple methods, e.g., based on the nearest-neighbor heuristic or linear models. None of the computationally complex neural methods was actually consistently better than already existing learning-based techniques, e.g., using matrix factorization or linear models. In our analysis, we discuss common issues in today’s research practice, which, despite the many papers that are published on the topic, have apparently led the field to a certain level of stagnation. 1

In recent years, algorithm research in the area of recommender systems has shifted from matrix factorization techniques and their latent factor models to neural approaches. However, given the proven power of latent factor models, some newer neural approaches incorporate them within more complex network architectures. One specific idea, recently put forward by several researchers, is to consider potential correlations between the latent factors, i.e., embeddings, by applying convolutions over the user-item interaction map. However, contrary to what is claimed in these articles, such interaction maps do not share the properties of images where Convolutional Neural Networks (CNNs) are particularly useful. In this work, we show through analytical considerations and empirical evaluations that the claimed gains reported in the literature cannot be attributed to the ability of CNNs to model embedding correlations, as argued in the original papers. Moreover, additional performance evaluations show that all of the examined recent CNN-based models are outperformed by existing non-neural machine learning techniques or traditional nearest-neighbor approaches. On a more general level, our work points to major methodological issues in recommender systems research.

In this article, we introduce the \dataset dataset, a collection of implicit interactions and impressions of movies and TV series from an Over-The-Top media service, which delivers its media contents over the Internet. The dataset is distinguished from other already available multimedia recommendation datasets by the availability of impressions, \idest the recommendations shown to the user, its size, and by being open-source. We describe the data collection process, the preprocessing applied, its characteristics, and statistics when compared to other commonly used datasets. We also highlight several possible use cases and research questions that can benefit from the availability of user impressions in an open-source dataset. Furthermore, we release software tools to load and split the data, as well as examples of how to use both user interactions and impressions in several common recommendation algorithms.

The development of continuously improved machine learning algorithms for personalized item ranking lies at the core of today's research in the area of recommender systems. Over the years, the research community has developed widely-agreed best practices for comparing algorithms and demonstrating progress with offline experiments. Unfortunately, we find this accepted research practice can easily lead to phantom progress due to the following reasons: limited reproducibility, comparison with complex but weak and non-optimized baseline algorithms, over-generalization from a small set of experimental configurations. To assess the extent of such problems, we analyzed 18 research papers published recently at top-ranked conferences. Only 7 were reproducible with reasonable effort, and 6 of them could often be outperformed by relatively simple heuristic methods, e.g., nearest neighbors. In this paper, we discuss these observations in detail, and reflect on the related fundamental problem of over-reliance on offline experiments in recommender systems research.

The main objective of exams consists in performing an assessment of students' expertise on a specific subject. Such expertise, also referred to as skill or knowledge level, can then be leveraged in different ways (e.g., to assign a grade to the students, to understand whether a student might need some support, etc.). Similarly, the questions appearing in the exams have to be assessed in some way before being used to evaluate students. Standard approaches to questions' assessment are either subjective (e.g., assessment by human experts) or introduce a long delay in the process of question generation (e.g., pretesting with real students). In this work we introduce R2DE (which is a Regressor for Difficulty and Discrimination Estimation), a model capable of assessing newly generated multiple-choice questions by looking at the text of the question and the text of the possible choices. In particular, it can estimate the difficulty and the discrimination of each question, as they are defined in Item Response Theory. We also present the results of extensive experiments we carried out on a real world large scale dataset coming from an e-learning platform, showing that our model can be used to perform an initial assessment of newly created questions and ease some of the problems that arise in question generation.

Selecting the right compiler optimisations has a severe impact on programs' performance. Still, the available optimisations keep increasing, and their effect depends on the specific program, making the task human intractable. Researchers proposed several techniques to search in the space of compiler optimisations. Some approaches focus on finding better search algorithms, while others try to speed up the search by leveraging previously collected knowledge. The possibility to effectively reuse previous compilation results inspired us toward the investigation of techniques derived from the Recommender Systems field. The proposed approach exploits previously collected knowledge and improves its characterisation over time. Differently from current state-of-the-art solutions, our approach is not based on performance counters but relies on Reaction Matching, an algorithm able to characterise programs looking at how they react to different optimisation sets. The proposed approach has been validated using two widely used benchmark suites, cBench and PolyBench, including 54 different programs. Our solution, on average, extracted 90% of the available performance improvement 10 iterations before current state-of-the-art solutions,which corresponds to 40% fewer compilations and performance tests to perform.

Deep learning techniques have become the method of choice for researchers working on algorithmic aspects of recommender systems. With the strongly increased interest in machine learning in general, it has, as a result, become difficult to keep track of what represents the state-of-the-art at the moment, e.g., for top-n recommendation tasks. At the same time, several recent publications point out problems in today's research practice in applied machine learning, e.g., in terms of the reproducibility of the results or the choice of the baselines when proposing new models.

This paper focuses on recommender systems based on item-item collaborative filtering (CF). Although research on item-based methods is not new, current literature does not provide any reliable insight on how to estimate confidence of recommendations. The goal of this paper is to fill this gap, by investigating the conditions under which item-based recommendations will succeed or fail for a specific user. We formalize the item-based CF problem as an eigenvalue problem, where estimated ratings are equivalent to the true (unknown) ratings multiplied by a user-specific eigenvalue of the similarity matrix. We show that the magnitude of the eigenvalue related to a user is proportional to the accuracy of recommendations for that user. We define a confidence parameter called the eigenvalue confidence index, analogous to the eigenvalue of the similarity matrix, but simpler to be computed. We also show how to extend the eigenvalue confidence index to matrix-factorization algorithms. A comprehensive set of experiments on five datasets show that the eigenvalue confidence index is effective in predicting, for each user, the quality of recommendations. On average, our confidence index is 3 times more correlated with MAP with respect to previous confidence estimates.

As of today, most movie recommendation services base their recommendations on collaborative filtering (CF) and/or content-based filtering (CBF) models that use metadata (e.g., genre or cast). In most video-on-demand and streaming services, however, new movies and TV series are continuously added. CF models are unable to make predictions in such a scenario, since the newly added videos lack interactions—a problem technically known as new item cold start (CS). Currently, the most common approach to this problem is to switch to a purely CBF method, usually by exploiting textual metadata. This approach is known to have lower accuracy than CF because it ignores useful collaborative information and relies on human-generated textual metadata, which are expensive to collect and often prone to errors. User-generated content, such as tags, can also be rare or absent in CS situations. In this paper, we introduce a new movie recommender system that addresses the new item problem in the movie domain by (i) integrating state-of-the-art audio and visual descriptors, which can be automatically extracted from video content and constitute what we call the movie genome ; (ii) exploiting an effective data fusion method named canonical correlation analysis , which was successfully tested in our previous works Deldjoo et al. (in: International Conference on Electronic Commerce and Web Technologies. Springer, Berlin, pp 34–45, 2016b ; Proceedings of the Twelfth ACM Conference on Recommender Systems. ACM, 2018b ), to better exploit complementary information between different modalities; (iii) proposing a two-step hybrid approach which trains a CF model on warm items (items with interactions) and leverages the learned model on the movie genome to recommend cold items (items without interactions). Experimental validation is carried out using a system-centric study on a large-scale, real-world movie recommendation dataset both in an absolute cold start and in a cold to warm transition; and a user-centric online experiment measuring different subjective aspects, such as satisfaction and diversity. Results show the benefits of this approach compared to existing approaches.

In this paper we propose a new dataset, i.e., the MMTF-14K multi-faceted dataset. It is primarily designed for the evaluation of video-based recommender systems, but it also supports the exploration of other multimedia tasks such as popularity prediction, genre classification and auto-tagging (aka tag prediction). The data consists of 13,623 Hollywood-type movie trailers, ranked by 138,492 users, generating a total of almost 12.5 million ratings. To address a broader community, metadata, audio and visual descriptors are also pre-computed and provided along with several baseline benchmarking results for uni-modal and multi-modal recommendation systems. This creates a rich collection of data for benchmarking results and which supports future development of this field.

We propose a multi-modal content-based movie recommender system that replaces human-generated metadata with content descriptions automatically extracted from the visual and audio channels of a video. Content descriptors improve over traditional metadata in terms of both richness (it is possible to extract hundreds of meaningful features covering various modalities) and quality (content features are consistent across different systems and immune to human errors). Our recommender system integrates state-of-the-art aesthetic and deep visual features as well as block-level and i-vector audio features. For fusing the different modalities, we propose a rank aggregation strategy extending the Borda count approach.

In recent years, more and more recommendation algorithms have been proposed that are based on time-ordered user interaction logs. Algorithms for session-based recommendation tasks are among the most prominent examples of such approaches.

The majority of research works in the field of collaborative filtering recommender systems is based on the assumption that the input to the recommendation algorithms is a matrix containing user-item interactions. In reality, however, the input often is a sequence of various types of user-item interactions that are recorded over time and where we can have multiple data points per user-item pair. These sequential logs contain a variety of useful information that can be leveraged in the recommendation process, e.g., to predict the immediate next action of a user or to detect short-term trends in the community. In this tutorial we review what we call sequence-aware recommenders, i.e., approaches that aim to exploit the information in sequential interaction logs for a variety of different purposes. We in particular focus on sequential and session-based recommendation techniques and discuss algorithmic proposals as well as evaluation challenges.

In the last years, there has been much attention given to the semantic gap problem in multimedia retrieval systems. Much effort has been devoted to bridge this gap by building tools for the extraction of high-level, semantics-based features from multimedia content, as low-level features are not considered useful because they deal primarily with representing the perceived content rather than the semantics of it.

Session-based recommendations are highly relevant in many modern on-line services (e.g. e-commerce, video streaming) and recommendation settings. Recently, Recurrent Neural Networks have been shown to perform very well in session-based settings. While in many session-based recommendation domains user identifiers are hard to come by, there are also domains in which user profiles are readily available. We propose a seamless way to personalize RNN models with cross-session information transfer and devise a Hierarchical RNN model that relays end evolves latent hidden states of the RNNs across user sessions. Results on two industry datasets show large improvements over the session-only RNNs.

Item-based recommender systems suggest products based on the similarities between items computed either from past user preferences (collaborative filtering) or from item content features (content-based filtering). Collaborative filtering has been proven to outperform content-based filtering in a variety of scenarios. However, in item cold-start, collaborative filtering cannot be used directly since past user interactions are not available for the newly added items. Hence, content-based filtering is usually the only viable option left.

Smart light technology offers new dynamic and interactive capabilities that extend the potential of traditional lighting systems of attracting people and affecting their mood, emotions, and behavior. Our research explores smart lights in the context of shop windows. The paper describes an extensive empirical study that has been performed for 5 weeks in a top-level fashion store located in one of the trendiest and crowed shopping areas in Milan (Italy), where we installed a sophisticate smart lighting system and created three different light configurations (static, dynamic, and interactive). The goal of the study is to explore the shopping experience in front of the shop in these different experimental conditions. Users' data were gathered in three ways: i) automatic collection of data from over >300.000 people passing by or stopping in front of the shop windows of the store; ii) live observation of approximately 600 passers-by; iii) questionnaires submitted to 62 persons in front of the shop. Users' data provide empirical evidence that interactive smart lights have the potential to increase the attractiveness of shopping windows, fostering engagement in passers-by, and in general improve the shopping experience in the brick-and-mortar stores.

This paper describes a contemporary concept store, offering a technology-rich blend of entertainment and interactivity targeted to help customers in their shopping experiences while shortening the time they waste queuing.

The fashion industry is one of the most flourishing fields for visual applications of IT. Due to the importance of the concept of look in fashion, the most advanced applications of computer graphics and sensing may fruitfully be exploited. The existence of low cost solutions in similar fields, such as the ones that empower the domestic video games market, suggest that analogous low cost solutions are viable and can foster innovation even in small and medium enterprises. In this paper the current state of development of vMannequin, a dynamic, user mimicking, user enacted virtual mannequin software solution, is presented. In order to allow users designing dress concepts, the application simulate the creation and fitting of clothes on virtual models. The interaction is sensor based, in order to both simplify the user interface, and create a richer involvement inside the application.

In this position paper, we take the experimental approach of putting algorithms aside, and reflect on what recommenders would be for people if they were not tied to technology. By looking at some of the shortcomings that current recommenders have fallen into and discussing their limitations from a human point of view, we ask the question: if freed from all limitations, what should, and what could, RecSys be? We then turn to the idea that life itself is the best recommender system, and that people themselves are the query. By looking at how life brings people in contact with options that suit their needs or match their preferences, we hope to shed further light on what current RecSys could be doing better. Finally, we look at the forms that RecSys could take in the future. By formulating our vision beyond the reach of usual considerations and current limitations, including business models, algorithms, data sets, and evaluation methodologies, we attempt to arrive at fresh conclusions that may inspire the next steps taken by the community of researchers working on RecSys.

A critical change has occurred in the status of context in recommender systems. In the past, context has been considered 'additional evidence'. This past picture is at odds with many present application domains, where user and item information is scarce. Such domains face continuous cold start conditions and must exploit session rather than user information. In this paper, we describe the `Contextual Turn?: the move towards context-driven recommendation algorithms for which context is critical, rather than additional. We cover application domains, algorithms that promise to address the challenges of context-driven recommendation, and the steps that the community has taken to tackle context-driven problems. Our goal is to point out the commonalities of context-driven problems, and urge the community to address the overarching challenges that context-driven recommendation poses.

other Share on Introduction to the Special Issue on Recommender System Benchmarking Authors: Paolo Cremonesi Politecnico di Milano Politecnico di MilanoView Profile , Alan Said Recorded Future Recorded FutureView Profile , Domonkos Tikk Gravity R&D, Hungary Gravity R&D, HungaryView Profile , Michelle X. Zhou Juji JujiView Profile Authors Info & Claims ACM Transactions on Intelligent Systems and TechnologyVolume 7Issue 3April 2016 Article No.: 38pp 1–4https://doi.org/10.1145/2870627Published:08 March 2016Publication History 2citation384DownloadsMetricsTotal Citations2Total Downloads384Last 12 Months12Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Recent years witnessed an explosion in the number and variety of data crowdsourcing initiatives. From OpenStreetMap to Amazon Mechanical Turk, developers and practitioners have been striving to create user interfaces able to effectively and efficiently support the creation, exploration, and analysis of crowdsourced information.

When choosing an hotel, a restaurant or a movie, many people rely on the reviews available on the Web. However, this huge amount of opinions make it difficult for users to have a comprehensive vision of the crowd judgments and to make an optimal decision. In this work we provide evidence that automatic text summarization of reviews can be used to design Web applications able to effectively reduce the decision making effort in domains where decisions are based upon the opinion of the crowd.

The StreetSmart Project develops information services that integrate multiple (touch and touchless) interaction paradigms on personal devices and large public displays. It exploits personalization techniques in order to offer new engaging user experiences involving large amounts of multimedia contents.

Evaluation is a cardinal issue in recommender systems; as in any technical discipline, it highlights to a large extent the problems that need to be solved by the field and, hence, leads the way for algorithmic research and development in the community. Yet, in the field of recommender systems, there still exists considerable disparity in evaluation methods, metrics and experimental designs, as well as a significant mismatch between evaluation methods in the lab and what constitutes an effective recommendation for real users and businesses. Even after the relevant quality dimensions have been defined, a clear evaluation protocol should be specified in detail and agreed upon, allowing for the comparison of results and experiments conducted by different authors. This would enable any contribution to the same problem to be incremental and add up on top of previous work, rather than grow sideways. The REDD 2014 workshop seeks to provide an informal forum to tackle such issues and to move towards better understood and shared evaluation methodologies, allowing one to leverage the efforts and the workforce of the academic community towards meaningful and relevant directions in real-world developments.

Cross-domain recommender systems aim to generate or enhance personalized recommendations in a target domain by exploiting knowledge (mainly user preferences) from other source domains. This may beneficial for generating better recommendations, e.g. mitigating the cold-start and sparsity problems in a target domain, and enabling personalized cross-selling for items from multiple domains. In this tutorial, we formalize the cross-domain recommendation problem, categorize and survey state of the art cross-domain recommender systems, discuss related evaluation issues, and outline future research directions on the topic.

Cross-domain recommender systems adopt different techniques to transfer learning from source domain to target domain in order to alleviate the sparsity problem and improve accuracy of recommendations. Traditional techniques require the two domains to be linked by shared characteristics associated to either users or items. In collaborative filtering (CF) this happens when the two domains have overlapping users or item (at least partially). Recently, Li et al. [7] introduced codebook transfer (CBT), a cross-domain CF technique based on co-clustering, and presented experimental results showing that CBT is able to transfer knowledge between non-overlapping domains. In this paper, we disprove these results and show that CBT does not transfer knowledge when source and target domains do not overlap.

The CrowdRec workshop brings together the recommender system community for discussion and exchange of ideas. Its goal is to allow the potential of human computation and crowdsourcing to be exploited fully and sustainably, leading to the development of improved recommendation and information filtering technologies. Currently, the complete range of possible intelligent contributions that recommender systems could elicit from users is under-explored, and its full extent is unknown. Critical questions addressed in the workshop include how to: formulate crowdtasks, match tasks with crowdmembers, ensure the quality of crowd input, and integrate feedback from the crowd in an optimal manner to improve recommendation. Further, crowdsourcing can also be exploited for system design and system evaluation.

Markovian Arrival Processes (MAPs) are a tractable class of point-processes useful to model correlated time series, such as those commonly found in network traces and system logs used in performance analysis and reliability evaluation. Marked MAPs (MMAPs) generalize MAPs by further allowing the modeling of multi-class traces, possibly with cross-correlation between multi-class arrivals. In this paper, we present analytical formulas to fit second-order acyclic MMAPs with an arbitrary number of classes. We initially define closed-form formulas to fit second-order MMAPs with two classes, where the underlying MAP is in canonical form. Our approach leverages forward and backward moments, which have recently been defined, but never exploited jointly for fitting. Then, we show how to sequentially apply these formulas to fit an arbitrary number of classes. Representative examples and trace-driven simulation using storage traces show the effectiveness of our approach for fitting empirical datasets.

In a number of domains of interest for recommender systems, items are characterized by constrained and variable "capacity": the same product or service can be consumed by a limited number of users and the possibility of item consumption depends on contextual circumstances (e.g., time). Our work explores recommenders in the context of these "bounded" domains. We consider online hotel booking as a case study, and investigates if and how "missing" items (hotels that eventually becomes unavailable for users' consumption) affect the quality of recommendations. The paper proposes a technique for defining "missing" items as "best items", and presents an articulated empirical research in which recommendations for hotel online booking are evaluated in different experimental conditions with a user centric approach involving 142 participants.

A Recommender System (RS) filters a large amount of information to identify the items that are likely to be more interesting and attractive to a user. Recommendations are inferred on the basis of different user profile characteristics, in most cases including explicit ratings on a sample of suggested elements. RS research highlights that profile length, i. e., the number of collected ratings, is positively correlated to the accuracy of recommendations, which is considered an important quality factor for RSs. Still, gathering ratings adds a burden on the user, which may negatively affect the UX. A design tension seems to exist, induced by two conflicting requirements -- to raise accuracy by increasing the profile length, and to make the profiling process smooth for the user by limiting the number of ratings. The paper presents a wide empirical study (1080 users involved) which explores this issue. Our work attempts to identify which of the two contrasting forces influenced by profile length -- recommendations accuracy and burden of the rating process - has stronger effects on the perceived quality of the UX with a RS.

The growth of the social web poses new challenges and opportunities for recommender systems. The goal of Recommender Systems (RSs) is to filter information from a large data set and to recommend to users only the items that are most likely to interest and/or appeal to them. The quality of a RS is typically defined in terms of different attributes, the principal ones being relevance, novelty, serendipity and global satisfaction. Most existing works evaluate quality of Recommender Systems in terms of statistical factors that are algorithmically measured. This paper aims to explore whether (i) algorithmic measures of RS quality are in accordance with user-based measure and (ii) the user perceived quality of a RS is affected by the number of movies rated by the user. For this purpose we designed, developed and tested a web recommender system, TheBestMovie4You (http://www.moviers.it), which allows us to collect questionnaires about the quality of recommendations. We made a questionnaire and gave it to 240 subjects and we wanted to have as wide a set of users as possible using social web. In a experiment we asked the users to choose five movies (short profile), in a second to choose ten (long profile). Our results show that statistical properties fail in fully describing the quality of algorithms, because with user-centered metrics we can outline an algorithm's features that otherwise could not be detected. The comparison between the two phases highlighted a difference only in three cases out of twenty.

One of the unresolved issues when designing a recommender system is the number of ratings -- i.e., the profile length -- that should be collected from a new user before providing recommendations. A design tension exists, induced by two conflicting requirements. On the one hand, the system must collect "enough"ratings from the user in order to learn her/his preferences and improve the accuracy of recommendations. On the other hand, gathering more ratings adds a burden on the user, which may negatively affect the user experience. Our research investigates the effects of profile length from both a subjective (user-centric) point of view and an objective (accuracy-based) perspective. We carried on an offline simulation with three algorithms, and a set of online experiments involving overall 960 users and four recommender algorithms, to measure which of the two contrasting forces influenced by the number of collected ratings -- recommendations relevance and burden of the rating process -- has stronger effects on the perceived quality of the user experience. Moreover, our study identifies the potentially optimal profile length for an explicit, rating based, and human controlled elicitation strategy.

Recommender Systems (RSs) help users search large amounts of digital contents and services by allowing them to identify the items that are likely to be more attractive or useful. RSs play an important persuasion role, as they can potentially augment the users’ trust towards in an application and orient their decisions or actions towards specific directions. This article explores the persuasiveness of RSs, presenting two vast empirical studies that address a number of research questions. First, we investigate if a design property of RSs, defined by the statistically measured quality of algorithms, is a reliable predictor of their potential for persuasion. This factor is measured in terms of perceived quality, defined by the overall satisfaction, as well as by how users judge the accuracy and novelty of recommendations. For our purposes, we designed an empirical study involving 210 subjects and implemented seven full-sized versions of a commercial RS, each one using the same interface and dataset (a subset of Netflix), but each with a different recommender algorithm. In each experimental configuration we computed the statistical quality (recall and F-measures) and collected data regarding the quality perceived by 30 users. The results show us that algorithmic attributes are less crucial than we might expect in determining the user’s perception of an RS’s quality, and suggest that the user’s judgment and attitude towards a recommender are likely to be more affected by factors related to the user experience. Second, we explore the persuasiveness of RSs in the context of large interactive TV services. We report a study aimed at assessing whether measurable persuasion effects (e.g., changes of shopping behavior) can be achieved through the introduction of a recommender. Our data, collected for more than one year, allow us to conclude that, (1) the adoption of an RS can affect both the lift factor and the conversion rate, determining an increased volume of sales and influencing the user’s decision to actually buy one of the recommended products, (2) the introduction of an RS tends to diversify purchases and orient users towards less obvious choices (the long tail), and (3) the perceived novelty of recommendations is likely to be more influential than their perceived accuracy. Overall, the results of these studies improve our understanding of the persuasion phenomena induced by RSs, and have implications that can be of interest to academic scholars, designers, and adopters of this class of systems.

In many commercial systems, the 'best bet' recommendations are shown, but the predicted rating values are not. This is usually referred to as a top-N recommendation task, where the goal of the recommender system is to find a few specific items which are supposed to be most appealing to the user. Common methodologies based on error metrics (such as RMSE) are not a natural fit for evaluating the top-N recommendation task. Rather, top-N performance can be directly measured by alternative methodologies based on accuracy metrics (such as precision/recall).

In this paper we evaluate the performance of different collaborative algorithms in cold-start situations, where the initial lack of ratings may affect the quality of the algorithms. The evaluation has been performed on the pay-per-view datasets collected by two IP-television providers over a period of several months. The analysis shows that item-based algorithms perform better with respect to SVD-based ones in the early stage of the cold-start problem. Moreover, the accuracy of SVD-based algorithms, when using few latent factors, decreases with the time-evolution of the dataset. On the contrary, the same algorithms used with a large-enough number of latent features increase their accuracy with time and may outperform the item-based algorithms.

Traditional approaches for capacity planning are based on queueing network models. However, modeling with queueing networks requires the knowledge of the service demands of each class of workloads at each device described in the model. In real systems, such service demands can be very difficult to measure. In this paper, we present an optimization-based technique to address the problem. The technique is formulated as a robust linear parameter estimation that can be used with both closed and open queueing network models. We consider the case where aggregate measurements (throughput and utilization) are available. Such measurements are typically much easier to obtain than the service demands. We present experimental results which prove the effectiveness of the constrained and robust linear estimation.

Among existing modeling techniques, queueing networks with "finite capacity regions" have largely proven to be effective in characterizing push-back effects and simultaneous resource possession in which a request holds more resources simultaneously. Queueing network models with finite capacity regions impose upper bounds on the number of jobs that can simultaneously reside in a set of service centers. For this reason they can be used to model application constraints. However, since they do not satisfy product-form assumptions, they are difficult to treat. In this paper we propose a novel approximate method for closed multiclass queueing networks containing finite capacity regions and shared constraints. Our approach is based on Norton's theorem for queueing networks where a region is replaced by a single flow equivalent service center (FESC). We propose a population-mix driven definition of FESCs service rates which provides increased accuracy with respect to existing methods. We solve the resulting non-product-form network with a new approximate variant of the convolution algorithm proposed in the paper. A comparison with simulation shows that the algorithm typically has a 4% approximation error.

Queuing network models of modern computing systems must consider a large number of components (e.g., Web servers, DB servers, application servers, firewall, routers, networks) and hundreds of customers with very different resource requirements. The complexity of such models makes the application of exact solution techniques prohibitively expensive, motivating research on approximate methods. This paper proposes an interpolation-matching framework that allows a unified view of approximate solution techniques for closed product-form queuing networks. Depending upon the interpolating functional form and the matching populations selected, a large versatile family of new approximations can be generated. It is shown that all the known approximation strategies, including Linearizer, are instances of the interpolation-matching framework. Furthermore, a new approximation technique, based on a third-order polynomial, is obtained using the interpolation-matching framework. The new technique is shown to be more accurate than other known methods.

Introduces queuing network models for the performance analysis of SPMD (single-program, multiple-data) applications executed on general-purpose parallel architectures such as MIMD (multiple-input, multiple data) and clusters of workstations. The models are based on the pattern of computation, communication and I/O operations of typical parallel applications. Analysis of the models leads to the definition of speedup surfaces which capture the relative influence of processors and I/O parallelism and show the effects of different hardware and software components on the performance. Since the parameters of the models correspond to measurable program and hardware characteristics, the models can be used to anticipate the performance behavior of a parallel application as a function of the target architecture (i.e. the number of processors, number of disks, I/O topology, etc.).

This paper introduces queuing network models for the performance analysis of SPMD applications executed on general-purpose parallel architectures such as MIMD and clusters of workstations. The models are based on the pattern of computation, communication, and I/O operations of typical parallel applications. Analysis of the models leads to the definition of speedup surfaces which capture the relative influence of processors and I/O parallelism and show the effects of different hardware and software components on the performance. Since the parameters of the models correspond to measurable program and hardware characteristics, the models can be used to anticipate the performance behavior of a parallel application as a function of the target architecture (i.e., number of processors, number of disks, I/O topology, etc).

The load generated by new types of communications services related to multimedia and video transmission is becoming one of the major sources of traffic in WAN networks. Modeling this type of load is a prerequisite for any performance study. We approach the load-characterization problem from a global point of view by analyzing a set of 20 video streams. We developed resource-, subject-, and scene-oriented characterizations of coded video streams. We have also implemented multidimensional data-analysis techniques and applied a "scene working set" approach, as well as static and dynamic video traffic models. We show that the behavior of a video sequence can be predicted with high accuracy by applying the scene working set technique. Applications of this technique for predicting bandwidth demands and allocating buffers in an ATM switch are also described in this study.