Deviant behavior in online social systems is a difficult problem to address. Consequences of deviance include driving off users and tarnishing the system's public image. We present an examination of these concepts in a popular online game, League of Legends. Using a large collection of game records and player-given feedback, we develop a metric, toxicity index, to identify deviant players. We then look at the effects of interacting with deviant players, including effects on retention. We find that toxic players have several significant predictive patterns, such as playing in more competitive game modes and playing with friends. We also show that toxic players drive away new players, but that experienced players are more resilient to deviant behavior. Based on our findings, we suggest methods to better identify and counteract the negative effects of deviance.

In a business setting, the customer value is crucial as it determines how much it is worth spending to acquire a particular customer. Viral marketing techniques leverages social ties among users to help advertising a particular product. Recently, as mobile devices (e.g., smart phones, GPS devices) became ubiquitous, location-based social networking websites (e.g., Gowalla, BrightKite, Foursquare) are getting more and more popular. Along with location-based social networking services being prominent, new kind of data came into play besides the traditional social networking data: (1) Spatial data: represents the users geo-locations, venues geo-locations and information about users visiting different venues. (2) Users Opinions data: represents how much a user likes the venues she visits (e.g., Alice visited restaurant A and gave it a rating of five over five). In this paper, we present PLUTUS; a framework that assists venues (e.g., restaurant, gym, shopping mall) owners in growing their business. To recommend the best set of customers, PLUTUS takes three main aspects into consideration: (1) Social aspect, (2) Spatial aspect, and (3) Users opinions aspect. To this end, PLUTUS proposes two main algorithms: (1) Profit Calculation: It is responsible of calculating the total profit that a user u may add to a venue v taking into account the social, spatial, and user opinions aspects. (2) Profit Maximization: This algorithm is used to maximize the total profit of a given venue. We evaluated PLUTUS using real data set extracted from an existing Location-based Social Networking website, Foursquare. The results show that Plutus achieves higher estimated profit and more efficient profit calculation than naive marketing algorithms.

One of the challenges for recommender systems is that users struggle to accurately map their internal preferences to external measures of quality such as ratings. We study two methods for supporting the mapping process: (i) reminding the user of characteristics of items by providing personalized tags and (ii) relating rating decisions to prior rating decisions using exemplars. In our study, we introduce interfaces that provide these methods of support. We also present a set of methodologies to evaluate the efficacy of the new interfaces via a user experiment. Our results suggest that presenting exemplars during the rating process helps users rate more consistently, and increases the quality of the data.

In this paper we address the problem of developing actionable quality models for Wikipedia, models whose features directly suggest strategies for improving the quality of a given article. We first survey the literature in order to understand the notion of article quality in the context of Wikipedia and existing approaches to automatically assess article quality. We then develop classification models with varying combinations of more or less actionable features, and find that a model that only contains clearly actionable features delivers solid performance. Lastly we discuss the implications of these results in terms of how they can help improve the quality of articles across Wikipedia.

Dedicated and productive members who actively contribute to community efforts are crucial to the success of online volunteer groups such as Wikipedia. What predicts member productivity? Do productive members stay longer? How does involvement in multiple projects affect member contribution to the community? In this paper, we analyze data from 648 WikiProjects to address these questions. Our results reveal two critical trade-offs in managing online volunteer groups. First, factors that increase member productivity, measured by the number of edits on Wikipedia articles, also increase likelihood of withdrawal from contributing, perhaps due to feelings of mission accomplished or burnout. Second, individual membership in multiple projects has mixed effects. It decreases the amount of work editors contribute to both the individual projects and Wikipedia as a whole. It increases withdrawal for each individual project yet reduces withdrawal from Wikipedia. We discuss how our findings expand existing theories to fit the online context and inform the design of new tools to improve online volunteer work.

Hybrid recommender systems --- systems using multiple algorithms together to improve recommendation quality --- have been well-known for many years and have shown good performance in recent demonstrations such as the NetFlix Prize. Modern hybridization techniques, such as feature-weighted linear stacking, take advantage of the hypothesis that the relative performance of recommenders varies by circumstance and attempt to optimize each item score to maximize the strengths of the component recommenders. Less attention, however, has been paid to understanding what these strengths and failure modes are. Understanding what causes particular recommenders to fail will facilitate better selection of the component recommenders for future hybrid systems and a better understanding of how individual recommender personalities can be harnessed to improve the recommender user experience. We present an analysis of the predictions made by several well-known recommender algorithms on the MovieLens 10M data set, showing that for many cases in which one algorithm fails, there is another that will correctly predict the rating.

Most recommender systems assume user ratings accurately represent user preferences. However, prior research shows that user ratings are imperfect and noisy. Moreover, this noise limits the measurable predictive power of any recommender system. We propose an information theoretic framework for quantifying the preference information contained in ratings and predictions. We computationally explore the properties of our model and apply our framework to estimate the efficiency of different rating scales for real world datasets. We then estimate how the amount of information predictions give to users is related to the scale ratings are collected on. Our findings suggest a tradeoff in rating scale granularity: while previous research indicates that coarse scales (such as thumbs up / thumbs down) take less time, we find that ratings with these scales provide less predictive value to users. We introduce a new measure, preference bits per second, to quantitatively reconcile this tradeoff.

This editorial introduction explains the motivation and origin of the TiiS special issue on Highlights of the Decade in Interactive Intelligent Systems and shows how its five articles exemplify the types of research contribution that TiiS aims to encourage and publish.

This article introduces the tag genome, a data structure that extends the traditional tagging model to provide enhanced forms of user interaction. Just as a biological genome encodes an organism based on a sequence of genes, the tag genome encodes an item in an information space based on its relationship to a common set of tags. We present a machine learning approach for computing the tag genome, and we evaluate several learning models on a ground truth dataset provided by users. We describe an application of the tag genome called Movie Tuner which enables users to navigate from one item to nearby items along dimensions represented by tags. We present the results of a 7-week field trial of 2,531 users of Movie Tuner and a survey evaluating users’ subjective experience. Finally, we outline the broader space of applications of the tag genome.

Tags help users understand a rich information space, by showing them specific text annotations for each item in the space and enabling them to search by these annotations. Often, however, users may wish to move from one item to other items that are similar overall, but that differ in key characteristics. For example, a user who loves Pulp Fiction might want to see a similar movie, but might be in a mood for a less "dark" movie. This paper introduces Movie Tuner, a novel interface that supports navigation from one item to nearby items along dimensions represented by tags. Movie Tuner is based on a data structure called the tag genome, which is described in separate work. The tag genome encodes each item's relationship to a common set of tags by applying machine learning algorithms to user-contributed content. The present paper discusses our design of Movie Tuner, including algorithms for navigating to new items and for suggesting tags for navigation. We present the results of a 7-week field trial of 2,531 users of Movie Tuner, and of a survey evaluating users' subjective experience.

Recommender systems research is being slowed by the difficulty of replicating and comparing research results. Published research uses various experimental methodologies and metrics that are difficult to compare. It also often fails to sufficiently document the details of proposed algorithms or the evaluations employed. Researchers waste time reimplementing well-known algorithms, and the new implementations may miss key details from the original algorithm or its subsequent refinements. When proposing new algorithms, researchers should compare them against finely-tuned implementations of the leading prior algorithms using state-of-the-art evaluation methodologies. With few exceptions, published algorithmic improvements in our field should be accompanied by working code in a standard framework, including test harnesses to reproduce the described results. To that end, we present the design and freely distributable source code of LensKit, a flexible platform for reproducible recommender systems research. LensKit provides carefully tuned implementations of the leading collaborative filtering algorithms, APIs for common recommender system use cases, and an evaluation framework for performing reproducible offline evaluations of algorithms. We demonstrate the utility of LensKit by replicating and extending a set of prior comparative studies of recommender algorithms --- showing limitations in some of the original results --- and by investigating a question recently raised by a leader in the recommender systems community on problems with error-based prediction evaluation.

LensKit is a new recommender systems toolkit aiming to be a platform for recommender research and education. It provides a common API for recommender systems, modular implementations of several collaborative filtering algorithms, and an evaluation framework for consistent, reproducible offline evaluation of recommender algorithms. In this demo, we will showcase the ease with which LensKit allows recommenders to be configured and evaluated.

This editorial introduction describes the aims and scope of the ACM Transactions on Interactive Intelligent Systems , explains how it aims to constitute a landmark addition to the publication landscape, and shows how the five articles in this inaugural issue fit into the journal's conception.

No abstract available.

The "wisdom of crowds" argument emphasizes the importance of diversity in online collaborations, such as open source projects and Wikipedia. However, decades of research on diversity in offline work groups have painted an inconclusive picture. On the one hand, the broader range of insights from a diverse group can lead to improved outcomes. On the other hand, individual differences can lead to conflict and diminished performance. In this paper, we examine the effects of group diversity on the amount of work accomplished and on member withdrawal behaviors in the context of WikiProjects. We find that increased diversity in experience with Wikipedia increases group productivity and decreases member withdrawal -- up to a point. Beyond that point, group productivity remains high, but members are more likely to withdraw. Strikingly, no such diminishing returns were observed for differences in member interest, which increases productivity and decreases member withdrawal in a linear fashion. Our results suggest that the low visibility of individual differences in online groups may allow them to harvest more of the benefits of diversity while bearing less of the cost. We discuss how our findings can inform further research of online collaboration.

Online social production communities allow efficient construction of valuable and high-quality information sources. To be successful, community members must be effective at collaboration, including makink collective decisions in the presence of disagreement. We examined over 100,000 decisions made by small working groups in Wikipedia, and analyzed how decision quality in these online groups is influenced by four group composition factors: the size of the group, how members were invited to the group, the prior experience of group members, and apparent bias shown by the group administrator. Our findings lead us to recommendations for designers of social production communities.

All new researchers face the daunting task of familiarizing themselves with the existing body of research literature in their respective fields. Recommender algorithms could aid in preparing these lists, but most current algorithms do not understand how to rate the importance of a paper within the literature, which might limit their effectiveness in this domain. We explore several methods for augmenting existing collaborative and content-based filtering algorithms with measures of the influence of a paper within the web of citations. We measure influence using well-known algorithms, such as HITS and PageRank, for measuring a node's importance in a graph. Among these augmentation methods is a novel method for using importance scores to influence collaborative filtering. We present a task-centered evaluation, including both an offline analysis and a user study, of the performance of the algorithms. Results from these studies indicate that collaborative filtering outperforms content-based approaches for generating introductory reading lists.

In this paper we introduce tag expression, a novel form of preference elicitation that combines elements from tagging and rating systems. Tag expression enables users to apply affect to tags to indicate whether the tag describes a reason they like, dislike, or are neutral about a particular item. We present a user interface for applying affect to tags, as well as a technique for visualizing the overall community's affect. By analyzing 27,773 tag expressions from 553 users entered in a 3-month period, we empirically evaluate our design choices. We also present results of a survey of 97 users that explores users' motivations in tagging and measures user satisfaction with tag expression.

Wikipedia has millions of articles, many of which receive little attention. One group of Wikipedians believes these obscure entries should be removed because they are uninteresting and neglected; these are the deletionists. Other Wikipedians disagree, arguing that this long tail of articles is precisely Wikipedia's advantage over other encyclopedias; these are the inclusionists. This paper looks at two overarching questions on the debate between deletionists and inclusionists: (1) What are the implications to the long tail of the evolving standards for article birth and death? (2) How is viewership affected by the decreasing notability of articles in the long tail? The answers to five detailed research questions that are inspired by these overarching questions should help better frame this debate and provide insight into how Wikipedia is evolving.

Many websites use tags as a mechanism for improving item metadata through collective user effort. Users of tagging systems often apply far more tags to an item than a system can display. These tags can range in quality from tags that capture a key facet of an item, to those that are subjective, irrelevant, or misleading. In this paper we explore tag selection algorithms that choose the tags that sites display. Based on 225,000 ratings and survey responses, we conduct offline analyses of 21 tag selection algorithms. We select the three best performing algorithms from our offline analysis, and deploy them live on the MovieLens website to 5,695 users for three months. Based on our results, we offer tagging system designers advice about tag selection algorithms.

While recommender systems tell users what items they might like, explanations of recommendations reveal why they might like them. Explanations provide many benefits, from improving user satisfaction to helping users make better decisions. This paper introduces tagsplanations, which are explanations based on community tags. Tagsplanations have two key components: tag relevance, the degree to which a tag describes an item, and tag preference, the user's sentiment toward a tag. We develop novel algorithms for estimating tag relevance and tag preference, and we conduct a user study exploring the roles of tag relevance and tag preference in promoting effective tagsplanations. We also examine which types of tags are most useful for tagsplanations.

Tagging has emerged as a powerful mechanism that enables users to find, organize, and understand online entities. Recommender systems similarly enable users to efficiently navigate vast collections of items. Algorithms combining tags with recommenders may deliver both the automation inherent in recommenders, and the flexibility and conceptual comprehensibility inherent in tagging systems. In this paper we explore tagommenders, recommender algorithms that predict users' preferences for items based on their inferred preferences for tags. We describe tag preference inference algorithms based on users' interactions with tags and movies, and evaluate these algorithms based on tag preference ratings collected from 995 MovieLens users. We design and evaluate algorithms that predict users' ratings for movies based on their inferred tag preferences. Our tag-based algorithms generate better recommendation rankings than state-of-the-art algorithms, and they may lead to flexible recommender systems that leverage the characteristics of items users find most important.

Search engines are among the most-used resources on the internet. However, even today's most successful search engines struggle to provide high quality search results. According to recent studies as many as 50 percent of web search sessions fail to find any relevant results for the searcher. Researchers have proposed social search techniques, in which early searchers provide feedback that is used to improve relevance for later searchers. In this paper we investigate foundational questions of social search. In particular, we directly assess the degree of agreement among users about the relevance ranking of search results. We developed a simulated search engine interface that systematically randomizes Google's normal relevance ordering of the items presented to users. Our results show that (a) people are biased toward items in the top of the search lists, even if the list is randomized; (b) people explicit feedback is not biased and (c) people's shared preferences do not always agree with Google's result order. These results suggest that social search techniques might improve the effectiveness of web search engines.

Wikipedia’s brilliance and curse is that any user can edit any of the encyclopedia entries. We introduce the notion of the impact of an edit, measured by the number of times the edited version is viewed. Using several datasets, including recent logs of all article views, we show that frequent editors dominate what people see when they visit Wikipedia, and that this domination is increasing. Similarly, using the same impact measure, we show that the probability of a typical article view being damaged is small but increasing, and we present empirically grounded classes of damage. Finally, we make policy recommendations for Wikipedia and other wikis in light of these findings.

Many online communities use tags - community selected words or phrases - to help people find what they desire. The quality of tags varies widely, from tags that capture akey dimension of an entity to those that are profane, useless, or unintelligible. Tagging systems must often select a subset of available tags to display to users due to limited screen space. Because users often spread tags they have seen, selecting good tags not only improves an individual's view of tags, it also encourages them to create better tags in the future. We explore implicit (behavioral) and explicit (rating) mechanisms for determining tag quality. Based on 102,056 tag ratings and survey responses collected from 1,039 users over 100 days, we offer simple suggestions to designers of online communities to improve the quality of tags seen by their users.

Member-maintained communities ask their users to perform tasks the community needs. From Slashdot, to IMDb, to Wikipedia, groups with diverse interests create community-maintained artifacts of lasting value (CALV) that support the group's main purpose and provide value to others. Said communities don't help members find work to do, or do so without regard to individual preferences, such as Slashdot assigning meta-moderation randomly. Yet social science theory suggests that reducing the cost and increasing the personal value of contribution would motivate members to participate more.We present SuggestBot, software that performs intelligent task routing (matching people with tasks) in Wikipedia. SuggestBot uses broadly applicable strategies of text analysis, collaborative filtering, and hyperlink following to recommend tasks. SuggestBot's intelligent task routing increases the number of edits by roughly four times compared to suggesting random articles. Our contributions are: 1) demonstrating the value of intelligent task routing in a real deployment; 2) showing how to do intelligent task routing; and 3) sharing our experience of deploying a tool in Wikipedia, which offered both challenges and opportunities for research.

Many small online communities would benefit from increased diversity or activity in their membership. Some communities run the risk of dying out due to lack of participation. Others struggle to achieve the critical mass necessary for diverse and engaging conversation. But what tools are available to these communities to increase participation? Our goal in this research was to spark contributions to the movielens.org discussion forum, where only 2% of the members write posts. We developed personalized invitations, messages designed to entice users to visit or contribute to the forum. In two field experiments, we ask (1) if personalized invitations increase activity in a discussion forum, (2) how the choice of algorithm for intelligently choosing content to emphasize in the invitation affects participation, and (3) how the suggestion made to the user affects their willingness to act. We find that invitations lead to increased participation, as measured by levels of reading and posting. More surprisingly, we find that invitations emphasizing the social nature of the discussion forum increase user activity, while invitations emphasizing other details of the discussion are less successful.

This full-day tutorial is designed to convey an up-to-date, active understanding of a representative set of current developments in recommender systems that will help the participants to conduct cutting-edge research and/or to work more effectively with the currently widespread recommendation technology.

Rapid and continuous growth of digital libraries, coupled with brisk advancements in technology, has driven users to seek tools and services that are not only customized to their specific needs, but are also helpful in keeping them stay abreast with the latest developments in their field. TechLens is a recommender system that learns about its users through implicit feedback, builds correlations among them, and uses that information to generate recommendations that match the user's profile. It gives users control over which parts of their profile of known citations are used in forming recommendations for new articles. This demonstration is a prototype that showcases some of the tools and services that TechLens offers to the users of digital libraries.

Many online communities are emerging that, like Wikipedia, bring people together to build community-maintained artifacts of lasting value (CALVs). Motivating people to contribute is a key problem because the quantity and quality of contributions ultimately determine a CALV's value. We pose two related research questions: 1) How does intelligent task routing---matching people with work---affect the quantity of contributions? 2) How does reviewing contributions before accepting them affect the quality of contributions? A field experiment with 197 contributors shows that simple, intelligent task routing algorithms have large effects. We also model the effect of reviewing contributions on the value of CALVs. The model predicts, and experimental data shows, that value grows more slowly with review before acceptance. It also predicts, surprisingly, that a CALV will reach the same final value whether contributions are reviewed before or after they are made available to the community.

Item-oriented Web sites maintain repositories of information about things such as books, games, or products. Many of these Web sites offer discussion forums. However, these forums are often disconnected from the rich data available in the item repositories. We describe a system, movie linking, that bridges a movie recommendation Web site and a movie-oriented discussion forum. Through automatic detection and an interactive component, the system recognizes references to movies in the forum and adds recommendation data to the forums and conversation threads to movie pages. An eight week observational study shows that the system was able to identify movie references with precision of .93 and recall of .78. Though users reported that the feature was useful, their behavior indicates that the feature was more successful at enriching the interface than at integrating the system.

One of the important challenges faced by designers of online communities is eliciting sufficent contributions from community members. Users in online communities may have difficulty either in finding opportunities to add value, or in understanding the value of their contributions to the community. Various social science theories suggest that showing users different perspectives on the value they add to the community will lead to differing amounts of contribution. The present study investigates a design augmentation for an existing community Web site that could benefit from additional contribution. The augmented interface includes individualized opportunities for contribution and an estimate of the value of each contribution to the community. The value is computed in one of four different ways: (1) value to self; (2) value to a small group the user has affinity with; (3) value to a small group the user does not have affinity with; and (4) value to the entire user community. The study compares the effectiveness of the different notions of value to 160 community members.

A tagging community's vocabulary of tags forms the basis for social navigation and shared expression.We present a user-centric model of vocabulary evolution in tagging communities based on community influence and personal tendency. We evaluate our model in an emergent tagging system by introducing tagging features into the MovieLens recommender system.We explore four tag selection algorithms for displaying tags applied by other community members. We analyze the algorithms 'effect on vocabulary evolution, tag utility, tag adoption, and user satisfaction.

In today's data-rich networked world, people express many aspects of their lives online. It is common to segregate different aspects in different places: you might write opinionated rants about movies in your blog under a pseudonym while participating in a forum or web site for scholarly discussion of medical ethics under your real name. However, it may be possible to link these separate identities, because the movies, journal articles, or authors you mention are from a sparse relation space whose properties (e.g., many items related to by only a few users) allow re-identification. This re-identification violates people's intentions to separate aspects of their life and can have negative consequences; it also may allow other privacy violations, such as obtaining a stronger identifier like name and address.This paper examines this general problem in a specific setting: re-identification of users from a public web movie forum in a private movie ratings dataset. We present three major results. First, we develop algorithms that can re-identify a large proportion of public users in a sparse relation space. Second, we evaluate whether private dataset owners can protect user privacy by hiding data; we show that this requires extensive and undesirable changes to the dataset, making it impractical. Third, we evaluate two methods for users in a public forum to protect their own privacy, suppression and misdirection. Suppression doesn't work here either. However, we show that a simple misdirection strategy works well: mention a few popular items that you haven't rated.

Online communities need regular maintenance activities such as moderation and data input, tasks that typically fall to community owners. Communities that allow all members to participate in maintenance tasks have the potential to be more robust and valuable. A key challenge in creating member-maintained communities is building interfaces, algorithms, and social structures that encourage people to provide high-quality contributions. We use Karau and Williams' collective effort model to predict how peer and expert editorial oversight affect members' contributions to a movie recommendation website and test these predictions in a field experiment with 87 contributors. Oversight increased both the quantity and quality of contributions while reducing antisocial behavior, and peers were as effective at oversight as experts. We draw design guidelines and suggest avenues for future work from our results.

Recommender systems have been shown to help users find items of interest from among a large pool of potentially interesting items. Influence is a measure of the effect of a user on the recommendations from a recommender system. Influence is a powerful tool for understanding the workings of a recommender system. Experiments show that users have widely varying degrees of influence in ratings-based recommender systems. Proposed influence measures have been algorithm-specific, which limits their generality and comparability. We propose an algorithm-independent definition of influence that can be applied to any ratings-based recommender system. We show experimentally that influence may be effectively estimated using simple, inexpensive metrics.

introduction Share on Introduction to the special section on recommender systems Authors: John Riedl View Profile , Paul Dourish View Profile Authors Info & Claims ACM Transactions on Computer-Human InteractionVolume 12Issue 3September 2005 pp 371–373https://doi.org/10.1145/1096737.1096738Published:01 September 2005Publication History 11citation1,428DownloadsMetricsTotal Citations11Total Downloads1,428Last 12 Months14Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Recommender systems have emerged in the past several years as an effective way to help people cope with the problem of information overload. One application in which they have become particularly common is in e-commerce, where recommendation of items can often help a customer find what she is interested in and, therefore can help drive sales. Unscrupulous producers in the never-ending quest for market penetration may find it profitable to shill recommender systems by lying to the systems in order to have their products recommended more often than those of their competitors. This paper explores four open questions that may affect the effectiveness of such shilling attacks: which recommender algorithm is being used, whether the application is producing recommendations or predictions, how detectable the attacks are by the operator of the system, and what the properties are of the items being attacked. The questions are explored experimentally on a large data set of movie ratings. Taken together, the results of the paper suggest that new ways must be used to evaluate and detect shilling attacks on recommender systems.

Recommender systems have been evaluated in many, often incomparable, ways. In this article, we review the key decisions in evaluating collaborative filtering recommender systems: the user tasks being evaluated, the types of analysis and datasets being used, the ways in which prediction quality is measured, the evaluation of prediction attributes other than quality, and the user-based evaluation of the system as a whole. In addition to reviewing the evaluation strategies used by prior researchers, we present empirical results from the analysis of various accuracy metrics on one content domain where all the tested metrics collapsed roughly into three equivalence classes. Metrics within each equivalency class were strongly correlated, while metrics from different equivalency classes were uncorrelated.

Recommender systems using collaborative filtering are a popular technique for reducing information overload and finding products to purchase. One limitation of current recommenders is that they are not portable. They can only run on large computers connected to the Internet. A second limitation is that they require the user to trust the owner of the recommender with personal preference data. Personal recommenders hold the promise of delivering high quality recommendations on palmtop computers, even when disconnected from the Internet. Further, they can protect the user's privacy by storing personal information locally, or by sharing it in encrypted form. In this article we present the new PocketLens collaborative filtering algorithm along with five peer-to-peer architectures for finding neighbors. We evaluate the architectures and algorithms in a series of offline experiments. These experiments show that Pocketlens can run on connected servers, on usually connected workstations, or on occasionally connected portable devices, and produce recommendations that are as good as the best published algorithms to date.

Recommender systems use people's opinions about items in an information domain to help people choose other items. These systems have succeeded in domains as diverse as movies, news articles, Web pages, and wines. The psychological literature on conformity suggests that in the course of helping people make choices, these systems probably affect users' opinions of the items. If opinions are influenced by recommendations, they might be less valuable for making recommendations for other users. Further, manipulators who seek to make the system generate artificially high or low recommendations might benefit if their efforts influence users to change the opinions they contribute to the recommender. We study two aspects of recommender system interfaces that may affect users' opinions: the rating scale and the display of predictions at the time users rate items. We find that users rate fairly consistently across rating scales. Users can be manipulated, though, tending to rate toward the prediction the system shows, whether the prediction is accurate or not. However, users can detect systems that manipulate predictions. We discuss how designers of recommender systems might react to these findings.

Recommender systems have changed the way people shop online. Recommender systems on wireless mobile devices may have the same impact on the way people shop in stores. We present our experience with implementing a recommender system on a PDA that is occasionally connected to the network. This interface helps users of the MovieLens movie recommendation service select movies to rent, buy, or see while away from their computer. The results of a nine month field study show that although there are several challenges to overcome, mobile recommender systems have the potential to provide value to their users today

In a world where the number of choices can be overwhelming, recommender systems help users find and evaluate items of interest. They do so by connecting users with information regarding the content of recommended items or the opinions of other individuals. Such systems have become powerful tools in domains such as electronic commerce, digital libraries, and knowledge management. In this paper, we address such systems and introduce a new class of recommender system called meta-recommenders. Meta-recommenders provide users with personalized control over the generation of a single recommendation list formed from a combination of rich data using multiple information sources and recommendation techniques. We discuss experiments conducted to aid in the design of interfaces for a meta-recommender in the domain of movies. We demonstrate that meta-recommendations fill a gap in the current design of recommender systems. Finally, we consider the challenges of building real-world, usable meta-recommenders across a variety of domains.

Collaborative filtering has proven to be valuable for recommending items in many different domains. In this paper, we explore the use of collaborative filtering to recommend research papers, using the citation web between papers to create the ratings matrix. Specifically, we tested the ability of collaborative filtering to recommend citations that would be suitable additional references for a target research paper. We investigated six algorithms for selecting citations, evaluating them through offline experiments against a database of over 186,000 research papers contained in ResearchIndex. We also performed an online experiment with over 120 users to gauge user opinion of the effectiveness of the algorithms and of the utility of such recommendations for common research tasks. We found large differences in the accuracy of the algorithms in the offline experiment, especially when balanced for coverage. In the online experiment, users felt they received quality recommendations, and were enthusiastic about the idea of receiving recommendations in this domain.

Recommender systems have become valuable resources for users seeking intelligent ways to search through the enormous volume of information available to them. One crucial unsolved problem for recommender systems is how best to learn about a new user. In this paper we study six techniques that collaborative filtering recommender systems can use to learn about new users. These techniques select a sequence of items for the collaborative filtering system to present to each new user for rating. The techniques include the use of information theory to select the items that will give the most value to the recommender system, aggregate statistics to select the items the user is most likely to have an opinion about, balanced techniques that seek to maximize the expected number of bits learned per presented item, and personalized techniques that predict which items a user will have an opinion about. We study the techniques thru offline experiments with a large pre-existing user data set, and thru a live experiment with over 300 users. We show that the choice of learning technique significantly affects the user experience, in both the user effort and the accuracy of the resulting predictions.

We present PolyLens, a new collaborative filtering recommender system designed to recommend items for groups of users, rather than for individuals. A group recommender is more appropriate and useful for domains in which several people participate in a single activity, as is often the case with movies and restaurants. We present an analysis of the primary design issues for group recommenders, including questions about the nature of groups, the rights of group members, social value functions for groups, and interfaces for displaying group recommendations. We then report on our PolyLens prototype and the lessons we learned from usage logs and surveys from a nine-month trial that included 819 users. We found that users not only valued group recommendations, but were willing to yield some privacy to get the benefits of group recommendations. Users valued an extension to the group recommender system that enabled them to invite non-members to participate, via email.

Recommender systems have been revolutionizing the way shoppers and information seekers find what they want. We will study some of the tremendous successes and spectacular failures of recommenders in E-commerce to understand the causes of the success or failure. We will leverage that understanding into a set of principles for successfully applying recommenders to business problems. Finally, we will study the economic and social forces that are shaping the evolution of recommenders, and peer into the crystal ball to glimpse the directions the technology will be going in the future.

Recommender systems apply knowledge discovery techniques to the problem of making personalized recommendations for information, products or services during a liveinteraction. These systems, especially the k-nearest neighbor collaborative ltering based ones, are achieving widespread success on the Web. The tremendous growth in the amountofavailable information and the number of visitors to Web sites in recentyears poses some key challenges for recommender systems. These are: producing high quality recommendations, performing many recommendations per second for millions of users and items and achieving high coverage in the face of data sparsity. In traditional collaborative ltering systems the amountofwork increases with the number of participants in the system. New recommender system technologies are needed that can quickly produce high quality recommendations, even for very large-scale problems. To address these issues we have explored item-based collaborative ltering techniques. Item-based techniques rst analyze the user-item matrix to identify relationships between dierent items, and then use these relationships to indirectly compute recommendations for users. In this paper we analyze dierent item-based recommendation generation algorithms. Welookinto dierenttechniques for computing item-item similarities (e.g., item-item correlation vs. cosine similarities between item vectors) and dierenttechniques for obtaining recommendations from them (e.g., weighted sum vs. regression model). Finally, weexperimentally evaluate our results and compare them to the basic k-nearest neighbor approach. Our experiments suggest that item-based algorithms provide dramatically better performance than user-based algorithms, while at the same time providing better quality than th...

Automated collaborative filtering (ACF) systems predict a person's affinity for items or information by connecting that person's recorded interests with the recorded interests of a community of people and sharing ratings between like-minded persons. However, current recommender systems are black boxes, providing no transparency into the working of the recommendation. Explanations provide that transparency, exposing the reasoning and data behind a recommendation. In this paper, we address explanation interfaces for ACF systems - how they should be implemented and why they should be implemented. To explore how, we present a model for explanations based on the user's conceptual model of the recommendation process. We then present experimental results demonstrating what components of an explanation are the most compelling. To address why, we present experimental evidence that shows that providing explanations can improve the acceptance of ACF systems. We also describe some initial explorations into measuring how explanations can improve the filtering performance of users.

Recommender systems apply statistical and knowledge discovery techniques to the problem of making product recommendations during a live customer interaction and they are achieving widespread success in E-Commerce nowadays. In this paper, we investigate several techniques for analyzing large-scale purchase and preference data for the purpose of producing useful recommendations to customers. In particular, we apply a collection of algorithms such as traditional data mining, nearest-neighbor collaborative filtering, and dimensionality reduction on two different data sets. The first data set was derived from the web-purchasing transaction of a large E-commerce company whereas the second data set was collected from MovieLens movie recommendation site. For the experimental purpose, we divide the recommendation generation process into three sub processes - representation of input data, neighborhood formation, and recommendation generation. We devise different techniques for different sub processes and apply their combinations on our data sets to compare for recommendation quality and performance.

Annotation of software artifacts is common in software development, and vital for software inspection. People viewing annotated artifacts encounter delocalization: they must understand various parts of an artifact (and their annotations) to understand the part they are viewing. We taxonomize delocalization within software systems into lateral delocalization (different items of the artifact within the same development phase), longitudinal delocalization (related items in different phases), and historical delocalization (successive versions of the same item). We report on a pilot study of code inspection with AnnoSpec, an inspection tool supporting visibility of laterally-delocalized annotations. Our results suggest that addressing delocalization may help people perform inspections more effectively.

Recommender systems are changing from novelties used by a few E-commerce sites, to serious business tools that are re-shaping the world of E-commerce.Many of the largest commerce Web sites are already using recommender systems to help their customers find products to purchase.A recommender system learns from a customer and recommends products that she will find most valuable from among the available products.In this paper we present an explanation of how recommender systems help Ecommerce sites increase sales, and analyze six sites that use recommender systems including several sites that use more than one recommender system.Based on the examples, we create a taxonomy of recommender systems, including the interfaces they present to customers, the technologies used to create the recommendations, and the inputs they need from customers.We conclude with ideas for new applications of recommender systems to E-commerce.

% & ' ( ) * !+ , -) ./ 0 1 32 -4 5 6 1 & ) * 7 98 ": <; '= ?> <; ": A@ B DC &E )F HG I8 KJ L 9E )F NM O7 "8 QP RE 6S T> AU B <V W8 ":

Computer-supported annotation of software systems and their documentation, including design documentation and source code, is a common and important software engineering activity. Annotated documentation is used in both formal software inspection and informal software maintenance. Viewers of annotated systems may understand the software more easily if annotations are visible not just from the annotated item itself but from other, related items. We propose a general framework for interconnecting annotatable items in software systems to achieve this visibility. We describe filtering and broadening rules that viewers can use to select the annotations they desire to see. We illustrate this framework in the context of object-oriented software system development.

Collaborative filtering systems help address information overload by usingthe opinions of users in a community to make personal recommendations fordocuments to each user. Many collaborative filtering systems have few useropinions relative to the large number of documents available. This sparsityproblem can reduce the utility of the filtering system by reducing thenumber of documents for which the system can make recommendations andadversely affecting the quality of recommendations. This paper defines andimplements a model for integrating content-based ratings into acollaborative filtering system. The filterbot model allows collaborativefiltering systems to address sparsity by tapping the strength of contentfiltering techniques. We identify and evaluate metrics for assessing theeffectiveness of filterbots specifically, and filtering system enhancementsin general. Finally, we experimentally validate the filterbot approach byshowing that even simple filterbots such as spell checking can increase theutility for users of sparsely populated collaborative filtering systems.Keywords Collaborative filtering, information filtering, content analysis,recommendation systems, social filtering, GroupLens Research, informationfiltering agents.

Traditional software inspection requires participants to meet together at the same time in the same place.Distributed, asynchronous inspection allows participants to conduct meetings independently of time and space, making inspection more convenient.We report on an industrial study that we have performed using a tool designed for distributed, asynchronous software inspection.Our e>..-perience suggests that distributed, asynchronous software inspection is feasible, and is a costeffective means of collaboration for geographically distributed work groups.

In information visualization, as the volume and complexity of the data increases, researchers require more powerful visualization tools that enable them to more effectively explore multidimensional datasets. In this paper, we discuss the general utility of a novel visualization spreadsheet framework. Just as a numerical spreadsheetenables exploration of numbers, a visualization spreadsheet enables exploration of visual forms of information. We show that the spreadsheet approach facilitates certain information visualization tasks that are more difficult using other approaches. Unlike traditional spreadsheets, which store only simple data elements and formulas in each cell, a visualization spreadsheetcell can hold anentire complex data set, selection criteria, viewing specifications, and other information needed for a full-fledged information visualization. Similarly, inter-cell operations are far more complex, stretching beyond simple arithmetic and string operations to encompass a range of domain-specific operators. We have built two prototype systems that illustrate some of these research issues. The underlying approach in our work allows domain experts to define new data types and data operations, and enables visualization experts to incorporate new visualizations, viewing parameters, and view operations. 1

Goals and content: The moring session will introduce the concepts of information filtering, develop a taxonomy of the techniques used, and take a detailed look at present and historical applications of collaborative filtering technology. The afternoon session will investigate design issues including algorithms for making recommendations, obtaining user ratings, privacy, communications, and data storage.

Tomorrow's multimedia applications will stress all parts of a computer system.To determine the computer resources needed to meet application demands we have developed a new capacity planning model that is based on application quality as perceived by the user.We have applied our model to a Distributed Interactive Simulation flight simulator called the Viiual Cockpit.We investigate the quality of the Vkttral Cockpit on existing networks and processors and predict the effects of high-speed networks and high-performance processors on Virtual Cockpit quality.We find processor performance is the current bottleneck in application quality for the Virtual Cockpit, but that higher-speed networks, such as ATM, will be needed to meet network requirements after two to three generations of processor improvement.

Information visualization faces challenges presented by the need to represent abstract data and the relationships within the data. Previously, we presented a system for visualizing similarities between a single DNA sequence and a large database of other DNA sequences (E.H. Chi et al., 1995). Similarity algorithms generate similarity information in textual reports that can be hundreds or thousands of pages long. Our original system visualized the most important variables from these reports. However, the biologists we work with found this system so useful they requested visual representations of other variables. We present an enhanced system for interactive exploration of this multivariate data. We identify a larger set of useful variables in the information space. The new system involves more variables, so it focuses on exploring subsets of the data. We present an interactive system allowing mapping of different variables to different axes, incorporating animation using a time axis, and providing tools for viewing subsets of the data. Detail-on-demand is preserved by hyperlinks to the analysis reports. We present three case studies illustrating the use of these techniques. The combined technique of applying a time axis with a 3D scatter plot and query filters to visualization of biological sequence similarity data is both powerful and novel.

Biological sequence similarity analysis presents visualization challenges, primarily because of the massive amounts of discrete, multi dimensional data. Genomic data generated by molecular biologists is analyzed by algorithms that search for similarity to known sequences in large genomic databases. The output from these algorithms can be several thousand pages of text, and is difficult to analyze because of its length and complexity. We developed and implemented a novel graphical representation for sequence similarity search results, which visually reveals features that are difficult to find in textual reports. The method opens new possibilities in the interpretation of this discrete, multidimensional data by enabling interactive investigation of the graphical representation.

In laboratories around the world, neuroscientists from diverse disciplines are exploring various aspects of brain structure. Because of the size of the domain, neuroscientists must specialize, making it difficult to fit results together, causing some research efforts to be duplicated because of lack of sharing of information. The authors have begun a long-term project to build a neuroscience research database for brain structure. One aspect of the database is the ability to visualize high-quality, high-resolution micrographs montaged together into 3-D structures as they were in the living brain. As demonstrated in this paper's analysis, realistic presentation of these visualizations across computer networks will stress current and proposed gigabit networks. Image compression can reduce network loads, but wide-spread use of the visualizations will still require networks capable of sustaining terabits per second of throughput.>

Collaborative filters help people make choices based on the opinions of other people. GroupLens is a system for collaborative filtering of netnews, to help people find articles they will like in the huge stream of available articles. News reader clients display predicted scores and make it easy for users to rate articles after they read them. Rating servers, called Better Bit Bureaus, gather and disseminate the ratings. The rating servers predict scores based on the heuristic that people who agreed in the past will probably agree again. Users can protect their privacy by entering ratings under pseudonyms, without reducing the effectiveness of the score prediction. The entire architecture is open: alternative software for news clients and Better Bit Bureaus can be developed independently and can interoperate with the components we have developed.

Several techniques have been used to reduce the performance impact of process synchronization in fine-grained multiprocessor systems. These existing techniques tend to have long synchronization times or high shared-bus use, or they require complex and expensive hardware. A new technique is presented that uses distributed hardware locking queues to reduce both contention and latency to the minimum values that can be obtained using a shared-bus. This technique is shown to require at most two shared-bus transactions, with one transaction being typical. The latency for process continuation after obtaining a lock is reduced to near zero. Barrier synchronization using this distributed mechanism requires only one shared-bus transaction per processor involved in the barrier. This new technique is scalable and applicable to both new architectures and to existing systems, and is less complex than other hardware solutions.

Many software engineering tasks have a synchronous component that requires the participants to assemble together at the same time and place. This approach is expensive in terms of traveling, scheduling and human resources. Existing computer tools mitigate these constraints by adding structure to the meeting, providing on-line document support, and distributing the participants over geographic boundaries. The constraint remains, however, that all participants participate at the same timeWe propose relaxing the time constraint in software engineering tasks to resolve issues non-concurrently, in effect reducing (and in some cases eliminating) the need for the synchronous meeting. We hypothesize that support for asynchrony will enable software engineering teams to work together as effectively in different times as in same time.We have chosen software inspection as our candidate software engineering task because it is well-understood, highly-structured, and widely-practiced. We have designed and developed a Collaborative Asynchronous Inspection of Software (CAIS) meeting prototype that supports the meeting part of inspection. CAIS allows participants to effectively "meet" even when separated by time zones and working schedules. We have conducted a pilot study comparing the manual and CAIS meetings and present our results and lessons learned.

SuiteSound, a programming environment with integrated support for multimedia, is discussed. SuiteSound is built in the Suite object-based system on a conventional UNIX operating system. SuiteSound objects incorporate multimedia by creating flows and filters. Flows are streams of multimedia data moving through a sequence of objects. They bridge the gap between objects representing the state of an entity at a discrete point in time and space and continuous media such as live audio or video. Filters are intermediate objects between the source and destination of a flow. They take flow as input, perform one of several operations such as multiplex-in, multiplex-out, gain control, or silence deletion on it, and send the resulting flow to its destination. In effect, they provide a virtual device interface for the application programmer that is uniform and independent of any physical device. The design and implementation of SuiteSound on the Sun SparcStation are described. Experiments performed to determine the network and CPU load of the sound tool are reviewed.>

A series of experiments is being conducted on the RAID distributed database system to study the performance and reliability implications of providing static and dynamic adaptability. The authors' studies of the cost of their adaptable implementation were conducted in the context of the concurrency controller and the replication controller. It is shown that adaptable implementations can be provided at costs comparable to those of special-purpose implementations. The experimentation with dynamic adaptability focuses on concurrency control. It is shown that dynamic adaptability can result in performance benefits and that system reconfiguration can be accomplished dynamically with less cost than stopping the system, performing reconfiguration, and then restarting the system. The authors' examination of the costs of providing greater data availability includes studying the replication control and atomicity control subsystems of RAID. The cost associated with increasing availability in an adaptable scheme of replication control and commit protocols is demonstrated.>

Experimentation with several methods of providing efficient communication facilities for distributed database systems is described. These studies give insight into the delays incurred by applications running on distributed systems. Five different mechanisms for local interprocess communications (two variations with message queues, named pipes, shared memory, and UDP sockets) have been implemented, compared, and analyzed. The most efficient of these is three times as fast as UDP for 1000-byte messages. Kernel-level software multicast and hardware multicast have also been implemented and their performance analyzed. The results show the significant advantage of using these techniques instead of using multiple sends and receives at the user level. The design of a facility that allows the dynamic addition of user-level protocols such as two-phase commit, clock synchronization, etc. to an operating system kernel is presented. The facility is based on a simple stack-based language that provides the functionality and security required.>

Adaptability is an essential tool for managing escalating software costs and to build high-reliability, high-performance systems. Algorithmic adaptability, which supports techniques for switching between classes of schedulers in distributed transaction systems, is modeled. RAID, an experimental system implemented to support experimentation in adaptability, is discussed. Adaptability features in RAID, including algorithmic adaptability, fault tolerance, and implementation techniques for an adaptable server-based design, are modeled.>

Raid, a robust and adaptable distributed database system for transaction processing, is described. Raid is a message-passing system, with server processes on each site. The servers manage concurrent processing, consistent replicated copies during site failures and atomic distributed commitment. A high-level, layered communications package provides a clean, location-independent interface between servers. The latest design of the communications package delivers messages via shared memory in a high-performance configuration in which several servers are linked into a single process. Raid provides the infrastructure to experimentally investigate various methods for supporting reliable distributed transaction processing. Measurements on transaction processing time and server CPU time are presented. Data and conclusions of experiments in three categories are also presented: communications software, consistent replicated copy control during site failures, and concurrent distributed checkpointing. A software tool for the evaluation of transaction processing algorithms in an operating system kernel is proposed.>

A model is presented for an adaptable system that allows online switching of classes of algorithms for database transaction processing. The basic idea is to identify conditions on the state of processing that will maintain consistency during the switch from one class to another. The classes of concurrency control algorithms and the formalism of history for transaction processing and serializability have been used to develop this research. In addition to the formalism, the precise conditions for switching digraph-serializable (DSR) algorithms have been given. This research is being applied to switching network partition protocols (conservative to optimistic), commit protocols, recovery block software, and has led towards the design of an adaptable and reconfigurable distributed database system. An experimental system called RAID has been implemented to test these ideas; it has been noted that adaptability provides for varying performance requirements and deals with failures of sites, transactions, and other components of the system.>

RAID is a robust and adaptable distributed system for transaction processing. It is a message-passing system, with server processes on each site. A high-level, layered communications package provides a clean, location independent interface between servers. RAID processes concurrent updates and retrievals on multiple sites. The servers manage concurrent processing, consistent replicated copies during site failures or network partitionings, and atomic distributed commitment. The latest version of the communications package is able to deliver messages in a high-performance configuration in which several servers are linked into a single process. RAID provides the infrastructure to investigate experimentally various methods for supporting reliable distributed-transaction processing. Experiments on handling site failure with partial replication, checkpointing, and alternative communications methods have been performed. Measurements on various aspects of RAID transaction processing performance are presented.>