With the increasing use of recommender systems in various application domains, many algorithms have been proposed for improving the accuracy of recommendations. Among various dimensions of recommender systems performance, long-tail (niche) recommendation performance remains an important challenge in large part because of the popularity bias of many existing recommendation techniques. In this study, we propose CORE, a cosine pattern–based technique, for effective long-tail recommendation. Comprehensive experimental results compare the proposed approach with a wide variety of classic, widely used recommendation algorithms and demonstrate its practical benefits in accuracy, flexibility, and scalability in addition to the superior long-tail recommendation performance. 1 History: Accepted by Ramaswamy Ramesh, Area Editor for Data Science & Machine Learning. Funding: This work was supported by the National Natural Science Foundation of China [Grants 72031001, 72072091, 72242101]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0194 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0194 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Contextual information has been widely recognized as an important modeling dimension in social sciences and in computing. In particular, the role of context has been recognized in enhancing recommendation results and retrieval performance. While a substantial amount of existing research has focused on context-aware recommender systems (CARS), many interesting problems remain under-explored. The CARS 2024 workshop provides a venue for presenting and discussing the important features of the next generation of CARS and application domains that may require the use of novel types of contextual information and cope with their dynamic properties in group recommendations and in online environments.

Contextual information has been widely recognized as an important modeling dimension in social sciences and in computing. In particular, the role of context has been recognized in enhancing recommendation results and retrieval performance. While a substantial amount of existing research has focused on context-aware recommender systems (CARS), many interesting problems remain under-explored. The CARS 2023 workshop provides a venue for presenting and discussing: the important features of the next generation of CARS; and application domains that may require the use of novel types of contextual information and cope with their dynamic properties in group recommendations and in online environments.

Contextual information has been widely recognized as an important modeling dimension in social sciences and in computing. In particular, the role of context has been recognized in enhancing recommendation results and retrieval performance. While a substantial amount of existing research has focused on context-aware recommender systems (CARS), many interesting problems remain under-explored. The CARS 2022 workshop provides a venue for presenting and discussing: the important features of the next generation of CARS; and application domains that may require the use of novel types of contextual information and cope with their dynamic properties in group recommendations and in online environments.

Numerical predictive modeling is widely used in different application domains. Although many modeling techniques have been proposed, and a number of different aggregate accuracy metrics exist for evaluating the overall performance of predictive models, other important aspects, such as the reliability (or confidence and uncertainty) of individual predictions, have been underexplored. We propose to use estimated absolute prediction error as the indicator of individual prediction reliability, which has the benefits of being intuitive and providing highly interpretable information to decision makers, as well as allowing for more precise evaluation of reliability estimation quality. As importantly, the proposed reliability indicator allows the reframing of reliability estimation itself as a canonical numeric prediction problem, which makes the proposed approach general-purpose (i.e., it can work in conjunction with any outcome prediction model), alleviates the need for distributional assumptions, and enables the use of advanced, state-of-the-art machine learning techniques to learn individual prediction reliability patterns directly from data. Extensive experimental results on multiple real-world data sets show that the proposed machine learning-based approach can significantly improve individual prediction reliability estimation as compared with a number of baselines from prior work, especially in more complex predictive scenarios.

Because of the accessibility of big data collections from consumers, products, and stores, advanced sales forecasting capabilities have drawn great attention from many businesses, especially those in retail, because of the importance of forecasting in decision making. Improvement of forecasting accuracy, even by a small percentage, may have a substantial impact on companies’ production and financial planning, marketing strategies, inventory controls, and supply chain management. Specifically, our research goal is to forecast the sales of each product in each store in the near future. Motivated by tensor factorization methodologies for context-aware recommender systems, we propose a novel approach called the advanced temporal latent factor approach to sales forecasting, or ATLAS for short, which achieves accurate and individualized predictions for sales by building a single tensor factorization model across multiple stores and products. Our contribution is a combination of a tensor framework (to leverage information across stores and products), a new regularization function (to incorporate demand dynamics), and extrapolation of the tensor into future time periods using state-of-the-art statistical (seasonal autoregressive integrated moving-average models) and machine-learning (recurrent neural networks) models. The advantages of ATLAS are demonstrated on eight product category data sets collected by Information Resources, Inc., where we analyze a total of 165 million weekly sales transactions of over 15,560 products from more than 1,500 grocery stores. Summary of Contribution: Sales forecasting has been a task of long-standing importance. Accurate sales forecasting provides critical managerial implications for companies’ decision making and operations. Improvement of forecasting accuracy may have a substantial impact on companies’ production planning, marketing strategies, inventory controls, and supply chain management, among other things. This paper proposes a novel computational (machine-learning-based) approach to sales forecasting and thus is positioned directly at the intersection of computing and business/operations research.

Contextual information has been widely recognized as an important modeling dimension both in social sciences and in computing. In particular, the role of context has been recognized in enhancing recommendation results and retrieval performance. While a substantial amount of existing research has focused on context-aware recommender systems (CARS), many interesting problems remain under-explored. The CARS 2021 workshop provides a venue for presenting and discussing: the important features of the next generation of CARS; and application domains that may require the use of novel types of contextual information and cope with their dynamic properties in group recommendations and in online environments.

Prior research has shown a robust effect of personalized product recommendations on user preference judgments for items. Specifically, the display of system-predicted preference ratings as item recommendations has been shown in multiple studies to bias users’ preference ratings after item consumption in the direction of the predicted rating. Top-N lists represent another common approach for presenting item recommendations in recommender systems. Through three controlled laboratory experiments, we show that top-N lists do not induce a discernible bias in user preference judgments. This result is robust, holding for both lists of personalized item recommendations and lists of items that are top-rated based on averages of aggregate user ratings. Adding numerical ratings to the list items does generate a bias, consistent with earlier studies. Thus, in contexts where preference biases are of concern to an online retailer or platform, top-N lists, without numerical predicted ratings, would be a promising format for displaying item recommendations.

Contextual information has been widely recognized as an important modeling dimension both in social sciences and in computing. In particular, the role of context has been recognized in enhancing recommendation results and retrieval performance. While a substantial amount of existing research has focused on context-aware recommender systems (CARS), many interesting problems remain under-explored. The CARS 2020 workshop provides a venue for presenting and discussing approaches for the next generation of CARS and application domains that may require the use of novel types of contextual information and cope with their dynamic properties in online environments.

We study the impact and interplay of social design features on the engagement behaviors toward user-generated content on Facebook business pages. By examining the introduction of the “Reactions” feature on Facebook, we aim to understand how the introduction of a new engagement feature affects the overall engagement activities and the use of existing engagement features. We found evidence of a positive effect of Reactions on overall engagement levels. Furthermore, the introduction of the Reactions feature had heterogeneous effects on the use of existing engagement features. Posts that received Reactions also ended up receiving more Likes and Comments than what they would have received before the feature change. However, the opposite is true for posts that received no Reactions, although the effect sizes were small. These effects were detected within the first four weeks after the feature introduction, and persisted after six months, indicating long-term structural changes in users’ engagement behaviors.

Contextual information has been widely recognized as an important modeling dimension both in social sciences and in computing. In particular, the role of context has been recognized in enhancing recommendation results and retrieval performance. While a substantial amount of existing research has focused context-aware recommender systems (CARS), many interesting problems remain under-explored. The CARS 2019 workshop provides a venue for presenting and discussing approaches for next generation of CARS and application domains that may require a variety of dimensions of contexts and cope with its dynamic properties.

User-system interaction in recommender systems involves three aspects: temporal browsing (viewing recommendation lists and/or searching/filtering), action (performing actions on recommended items, e.g., clicking, consuming) and inaction (neglecting or skipping recommended items). Modern recommenders build machine learning models from recordings of such user interaction with the system, and in doing so they commonly make certain assumptions (e.g., pairwise preference orders, independent or competitive probabilistic choices, etc.). In this paper, we set out to study the effects of these assumptions along three dimensions in eight different single models and three associated hybrid models on a user browsing data set collected from a real-world recommender system application. We further design a novel model based on recurrent neural networks and multi-task learning, inspired by Decision Field Theory, a model of human decision making. We report on precision, recall, and MAP, finding that this new model outperforms the others.

Temporally, users browse and interact with items in recommender systems. However, for most systems, the majority of the displayed items do not elicit any action from users. In other words, the user-system interaction process includes three aspects: browsing, action, and inaction. Prior recommender systems literature has focused more on actions than on browsing or inaction. In this work, we deployed a field survey in a live movie recommender system to interpret what inaction means from both the user's and the system's perspective, guided by psychological theories of human decision making. We further systematically study factors to infer the reasons of user inaction and demonstrate with offline data sets that this descriptive and predictive inaction model can provide benefits for recommender systems in terms of both action prediction and recommendation timing.

Current recommender systems often show the same most-highly recommended items again and again ignoring the feedback that users neither rate nor click on those items. We conduct an online field experiment to test two ways of manipulating top-N recommendations with the goal of improving user experience: cycling the top-N recommendation based on their past presentation and serpentining the top-N list mixing the best items into later recommendation requests. We find interesting tensions between opt-outs and activities, user perceived accuracy and freshness. Cycling within the same session might be a "love it or hate it" recommender property because users in it have a higher opt-out rate but engage in more activities. Cycling across sessions and serpentining increase user activities without significantly affecting opt-out rates. Users perceive more change and freshness but less accuracy and familiarity. Combining cycling and serpentining does not work as well as each individual manipulation separately. These two ways of manipulations on top-N list demonstrate some attractive properties but also call for innovative approaches to overcome their potential costs.

In this paper, we summarize VAMS 2017 - a workshop on value-aware and multistakeholder recommendation co-located with RecSys 2017. The workshop encouraged forward-thinking papers in this new area of recommender systems research and obtained a diverse set of responses ranging from application results to research overviews.

The purpose of recommenders is often summarized as "help the users find relevant items", and the predominant operationalization of this goal has been to focus on the ability to numerically estimate the users' preferences for unseen items or to provide users with item lists ranked in accordance to the estimated preferences. This dominant, albeit narrow, view of the recommendation problem has been tremendously helpful in advancing research in different ways, e.g., through the establishment of standardized evaluation procedures and metrics. In reality, recommender systems can serve a variety of purposes from the point of view of both consumers and providers. Most of the purposes, however, are significantly underexplored, even though many of them are arguably more aligned with the real-world expectations for recommenders than our current predominant paradigm. Therefore, it is important to revisit our conceptualizations of the potential goals of recommenders and their operationalization as research problems. In this paper, we discuss a framework of recommendation goals and purposes and highlight possible future directions and challenges related to the operationalization of such alternative problem formulations.

Recommendation stability measures the extent to which a recommendation algorithm provides predictions that are consistent with each other. Several approaches have been proposed in prior work to defining, measuring, and improving the stability of recommendation algorithms. Previous studies have focused primarily on understanding and evaluating recommendation stability in prediction-oriented settings, i.e., recommendation settings where it is crucial to provide the precise prediction of a user’s preference rating for an item. However, the research literature has been largely silent on the topic of recommendation stability in other important types of settings, such as classification-oriented (i.e., where it is important to accurately classify the item as relevant versus irrelevant, without having to quantify the user’s preference precisely), ranking-oriented (i.e., where it is important to provide accurate relative ranking of items to users), or top-K oriented (i.e., where it is important to suggest K items that are most appealing to the user). Therefore, this paper builds on prior work by generalizing the notion of stability to a broader set of recommendation settings and developing corresponding stability metrics. The paper also provides a comprehensive empirical analysis of classification, ranking, and top-K stability performance of popular recommender algorithms on real-world rating data sets under a variety of settings.

This paper focuses on the measure of recommendation stability, which reflects the consistency of recommender system predictions. Stability is a desired property of recommendation algorithms and has important implications on users' trust and acceptance of recommendations. Prior research has reported that some popular recommendation algorithms can suffer from a high degree of instability. In this study, we explore two scalable, general-purpose meta-algorithmic approaches-based on bagging and iterative smoothing-that can be used in conjunction with different traditional recommendation algorithms to improve their stability. Our experimental results on real-world rating data demonstrate that both approaches can achieve substantially higher stability as compared to the original recommendation algorithms. Furthermore, perhaps as importantly, the proposed approaches not only do not sacrifice the predictive accuracy in order to improve recommendation stability, but are actually able to provide additional accuracy improvements.

Recommender systems are being used to help users find relevant items from a large set of alternatives in many online applications. Most existing recommendation techniques have focused on improving recommendation accuracy; however, diversity of recommendations has also been increasingly recognized in research literature as an important aspect of recommendation quality. This paper proposes several optimization-based approaches for improving aggregate diversity of top-N recommendations, including a greedy maximization heuristic, a graph-theoretic approach based on maximum flow or maximum bipartite matching computations, and an integer programming approach. The proposed approaches are evaluated using real-world movie rating data sets and demonstrate substantial improvements in both diversity and accuracy as compared to the recommendation reranking approaches, which have been introduced in prior literature for the purposes of diversity improvement and were used for baseline comparisons in our study. The paper also discusses the computational complexity and the scalability of the proposed approaches, as well as the potential directions for future work.

CARS 2012 builds upon the success of the three previous editions held in conjunction with the 3rd to 5th ACM Conferences on Recommender Systems from 2009 to 2011. The 1st CARS Workshop was held in New York, NY, USA, whereas Barcelona, Spain, was home of the 2nd CARS Workshop in 2010. In 2011, the 3rd CARS workshop was held in Chicago, IL, USA.

Recommender systems are becoming increasingly important to individual users and businesses for providing personalized recommendations. However, while the majority of algorithms proposed in recommender systems literature have focused on improving recommendation accuracy (as exemplified by the recent Netflix Prize competition), other important aspects of recommendation quality, such as the diversity of recommendations, have often been overlooked. In this paper, we introduce and explore a number of item ranking techniques that can generate substantially more diverse recommendations across all users while maintaining comparable levels of recommendation accuracy. Comprehensive empirical evaluation consistently shows the diversity gains of the proposed techniques using several real-world rating data sets and different rating prediction algorithms.

The article explores stability as a new measure of recommender systems performance. Stability is defined to measure the extent to which a recommendation algorithm provides predictions that are consistent with each other. Specifically, for a stable algorithm, adding some of the algorithm’s own predictions to the algorithm’s training data (for example, if these predictions were confirmed as accurate by users) would not invalidate or change the other predictions. While stability is an interesting theoretical property that can provide additional understanding about recommendation algorithms, we believe stability to be a desired practical property for recommender systems designers as well, because unstable recommendations can potentially decrease users’ trust in recommender systems and, as a result, reduce users’ acceptance of recommendations. In this article, we also provide an extensive empirical evaluation of stability for six popular recommendation algorithms on four real-world datasets. Our results suggest that stability performance of individual recommendation algorithms is consistent across a variety of datasets and settings. In particular, we find that model-based recommendation algorithms consistently demonstrate higher stability than neighborhood-based collaborative filtering techniques. In addition, we perform a comprehensive empirical analysis of many important factors (e.g., the sparsity of original rating data, normalization of input data, the number of new incoming ratings, the distribution of incoming ratings, the distribution of evaluation data, etc.) and report the impact they have on recommendation stability.

CARS 2011 builds upon the success of the two previous editions held in conjunction with the 3rd and 4th ACM Conferences on Recommender Systems in 2009 and 2010. The first CARS Workshop was held in New York, NY, USA (2009), and Barcelona, Spain, was the home of the second CARS Workshop in 2010.

CARS and CAMRa were organized under the Context-awareness in Recommendation Systems special event and gathered academic researchers as well as industrial practitioners in a workshop and challenge.

The paper introduces stability as a new measure of the recommender systems performance. In general, we define a recommendation algorithm to be "stable" if its predictions for the same items are consistent over a period of time, assuming that any new ratings that have been submitted to the recommender system over the same period of time are in complete agreement with system's prior predictions. In this paper, we advocate that stability should be a desired property of recommendation algorithms, because unstable recommendations can lead to user confusion and, therefore, reduce trust in recommender systems. Furthermore, we empirically evaluate stability of several popular recommendation algorithms. Our results suggest that model-based recommendation techniques demonstrate higher stability than memory-based collaborative filtering heuristics. We also find that the stability measure for recommendation techniques is influenced by many factors, including the sparsity of the initial rating data, the number of new incoming ratings (representing the length of the time period over which the stability is being measured), the distribution of the newly added rating values, and the rating normalization procedures employed by the recommendation algorithms.

No abstract available.

The importance of contextual information has been recognized by researchers and practitioners in many disciplines, including e-commerce personalization, information retrieval, ubiquitous and mobile computing, data mining, marketing, and management. While a substantial amount of research has already been performed in the area of recommender systems, most existing approaches focus on recommending the most relevant items to users without taking into account any additional contextual information, such as time, location, or the company of other people (e.g., for watching movies or dining out). In this chapter we argue that relevant contextual information does matter in recommender systems and that it is important to take this information into account when providing recommendations. We discuss the general notion of context and how it can be modeled in recommender systems. Furthermore, we introduce three different algorithmic paradigms – contextual prefiltering, post-filtering, and modeling – for incorporating contextual information into the recommendation process, discuss the possibilities of combining several contextaware recommendation techniques into a single unifying approach, and provide a case study of one such combined approach. Finally, we present additional capabilities for context-aware recommenders and discuss important and promising directions for future research.

Organizations and firms are capturing increasingly more data about their customers, suppliers, competitors, and business environment. Most of this data is multi-attribute (multi-dimensional) and temporal in nature. Data mining and business intelligence techniques are often used to discover patterns in such data; however, mining temporal relationships typically is a complex task. We propose a new data analysis and visualization technique for representing trends in multi-attribute temporal data using a clustering-based approach. We introduce C-TREND, a system that implements the temporal cluster graph construct, which maps multi-attribute temporal data to a two-dimensional directed graph that identifies trends in dominant data types over time. In this paper, we present our temporal clustering-based technique, discuss its algorithmic implementation and performance, demonstrate applications of the technique by analyzing data on wireless networking technologies and baseball batting statistics, and introduce a set of metrics for further analysis of discovered trends.

The need to validate large amounts of data with the help of the domain expert arises naturally in many data-intensive applications, including data mining, data stream, and database-related applications. This paper presents a general validation approach that generalizes different expert-driven validation methods developed for specialized validation problems. In particular, we model the validation process as a sequence of validation operators, explore various properties of such sequences, and present theoretical results that provide for better understanding of the validation process. We also address the problem of selecting the best validation sequence among the class of equivalent sequence permutations. We demonstrate that this optimization problem is NP-hard and present two heuristic algorithms for improving validation sequences.

This paper presents an overview of the field of recommender systems and describes the current generation of recommendation methods that are usually classified into the following three main categories: content-based, collaborative, and hybrid recommendation approaches. This paper also describes various limitations of current recommendation methods and discusses possible extensions that can improve recommendation capabilities and make recommender systems applicable to an even broader range of applications. These extensions include, among others, an improvement of understanding of users and items, incorporation of the contextual information into the recommendation process, support for multicriteria ratings, and a provision of more flexible and less intrusive types of recommendations.

The article presents a multidimensional (MD) approach to recommender systems that can provide recommendations based on additional contextual information besides the typical information on users and items used in most of the current recommender systems. This approach supports multiple dimensions, profiling information, and hierarchical aggregation of recommendations. The article also presents a multidimensional rating estimation method capable of selecting two-dimensional segments of ratings pertinent to the recommendation context and applying standard collaborative filtering or other traditional two-dimensional rating estimation techniques to these segments. A comparison of the multidimensional and two-dimensional rating estimation approaches is made, and the tradeoffs between the two are studied. Moreover, the article introduces a combined rating estimation method, which identifies the situations where the MD approach outperforms the standard two-dimensional approach and uses the MD approach in those situations and the standard two-dimensional approach elsewhere. Finally, the article presents a pilot empirical study of the combined approach, using a multidimensional movie recommender system that was developed for implementing this approach and testing its performance.

The problem of analyzing microarray data became one of important topics in bioinformatics over the past several years, and different data mining techniques have been proposed for the analysis of such data. In this paper, we propose to use association rule discovery methods for determining associations among expression levels of different genes. One of the main problems related to the discovery of these associations is the scalability issue. Microarrays usually contain very large numbers of genes that are sometimes measured in 10,000s. Therefore, analysis of such data can generate a very large number of associations that can often be measured in millions. The paper addresses this problem by presenting a method that enables biologists to evaluate these very large numbers of discovered association rules during the post-analysis stage of the data mining process. This is achieved by providing several rule evaluation operators, including rule grouping, filtering, browsing, and data inspection operators, that allow biologists to validate multiple individual gane regulation patterns at a time. By iteratively applying these operators, biologists can explore a significant part of all the initially generated rules in an acceptable period of time and thus answer biological questions that are of a particular interest to him or her. To validate our method, we tested our system on the microarray data pertaining to the studies of environmental hazards and their influence of gane expression processes. As a result, we managed to answer several questions that were of interest to the biologists that had collected this data.

This paper presents an architecture of the e-Butler service, i.e. a personalized consumer-centric infomediary delivering a broad range of shopping-related services to consumers. In order to provide these services effectively, e-Butler should be "intelligent". This paper describes how a certain level of intelligence can be achieved through the utilization of a broad range of technologies, including data mining, personalization, profiling, recommender systems, data warehousing, Web proxy servers, data monitoring and XML-based technologies, that are integrated into a unique architecture of e-Butler.

Gediminas Adomavicius New York University [email protected] In many applications, ranging from recommender systems to one-to-one marketing to Web browsing, it is important to build personalized profiles of individual users from their transactional histories. These profiles describe individual behavior of users and can be specified with sets of rules learned from user transactional histories using various data mining techniques. Since many discovered rules can be spurious, irrelevant, or trivial, one of the main problems is how to perform post-analysis of the discovered rules, i.e., how to validate customer profiles by separating “good” rules from the “bad.” This paper presents a method for validating such rules with an explicit participation of a human expert