Abstract Using off‐the‐shelf search technology provides a single point of access into library resources, but we found that such commercial systems are not entirely satisfactory for the academic library setting. In response to this, Oregon State University (OSU) Libraries designed and deployed LibraryFind, a metasearch system. We conducted a usability experiment comparing LibraryFind, the OSU Libraries Web site, and Google Scholar. Each participant used all three search systems in a controlled setting, and we recorded their behavior to determine the effectiveness and efficiency of each search system. In this article, we focus on understanding what factors are important to undergraduates in choosing their primary academic search system for class assignments. Based on a qualitative and quantitative analysis of the results, we found that mimicking commercial Web search engines is an important factor to attract undergradu‐ates; however, when undergraduates use these kinds of search engines, they expect similar performance to Web search engines, including factors such as relevance, speed, and the availability of a spell checker. They also expected to be able to find out what kinds of content and materials are available in a system. Participants' prior experience using academic search systems also affected their expectations of a new system.

There has been little research into how end users might be able to communicate advice to machine learning systems. If this resource--the users themselves--could somehow work hand-in-hand with machine learning systems, the accuracy of learning systems could be improved and the users' understanding and trust of the system could improve as well. We conducted a think-aloud study to see how willing users were to provide feedback and to understand what kinds of feedback users could give. Users were shown explanations of machine learning predictions and asked to provide feedback to improve the predictions. We found that users had no difficulty providing generous amounts of feedback. The kinds of feedback ranged from suggestions for reweighting of features to proposals for new features, feature combinations, relational features, and wholesale changes to the learning algorithm. The results show that user feedback has the potential to significantly improve machine learning systems, but that learning algorithms need to be extended in several ways to be able to assimilate this feedback.

Helping computer users rapidly locate files in their folder hierarchies has become an important research topic in today's intelligent user interface design. This paper reports on FolderPredictor, a software system that can reduce the cost of locating files in hierarchical folders. FolderPredictor applies a cost-sensitive prediction algorithm to the user's previous file access information to predict the next folder that will be accessed. Experimental results show that, on average, FolderPredictor reduces the cost of locating a file by 50%. Another advantage of FolderPredictor is that it does not require users to adapt to a new interface, but rather meshes with the existing interface for opening files on the Windows platform.

The TaskTracer system seeks to help multi-tasking users manage the resources that they create and access while carrying out their work activities. It does this by associating with each user-defined activity the set of files, folders, email messages, contacts, and web pages that the user accesses when performing that activity. The initial TaskTracer system relies on the user to notify the system each time the user changes activities. However, this is burdensome, and users often forget to tell TaskTracer what activity they are working on. This paper introduces TaskPredictor, a machine learning system that attempts to predict the user’s current activity. TaskPredictor has two components: one for general desktop activity and another specifically for email. TaskPredictor achieves high prediction precision by combining three techniques: (a) feature selection via mutual information, (b) classification based on a confidence threshold, and (c) a hybrid design in which a Naive Bayes classifier estimates the classification confidence but where the actual classification decision is made by a support vector machine. This paper provides experimental results on data collected from Task-Tracer users.

This paper reports on TaskTracer --- a software system being designed to help highly multitasking knowledge workers rapidly locate, discover, and reuse past processes they used to successfully complete tasks. The system monitors users' interaction with a computer, collects detailed records of users' activities and resources accessed, associates (automatically or with users' assistance) each interaction event with a particular task, enables users to access records of past activities and quickly restore task contexts. We present a novel Publisher-Subscriber architecture for collecting and processing users' activity data, describe several different user interfaces tried with TaskTracer, and discuss the possibility of applying machine learning techniques to recognize/predict users' tasks.

Today's university library has many digitally accessible resources, both indexes to content and considerable original content. Using off-the-shelf search technology provides a single point of access into library resources, but we have found that such full-text indexing technology is not entirely satisfactory for library searching.

Recommendation systems analyze user preferences and recommend items to a user by predicting the user's preference for those items. Among various kinds of recommendation methods, collaborative filtering (CF) has been widely used and successfully applied to practical applications. However, collaborative filtering has two inherent problems: data sparseness and the cold-start problems. In this paper, we propose a method of integrating additional feature information of users and items into CF to overcome the difficulties caused by sparseness and improve the accuracy of recommendation. Several experimental results that show the effectiveness of the proposed method are also presented.

Collaborative Filtering (CF) systems have been researched for over a decade as a tool to deal with information overload. At the heart of these systems are the algorithms which generate the predictions and recommendations.In this article we empirically demonstrate that two of the most acclaimed CF recommendation algorithms have flaws that result in a dramatically unacceptable user experience.In response, we introduce a new Belief Distribution Algorithm that overcomes these flaws and provides substantially richer user modeling. The Belief Distribution Algorithm retains the qualities of nearest-neighbor algorithms which have performed well in the past, yet produces predictions of belief distributions across rating values rather than a point rating value.In addition, we illustrate how the exclusive use of the mean absolute error metric has concealed these flaws for so long, and we propose the use of a modified Precision metric for more accurately evaluating the user experience.

This paper presents the SERF (System for Electronic Recommendation Filtering) which is a collaborative filtering system that recommends context-sensitive, high-quality information sources for document search. Collaborative filtering systems remove the limitation of traditional content-based search by using individual's ratings to evaluate and recommend information sources. SERF uses collaborative filtering algorithms to predict the relevance and quality of each document with respect to each particular user and their specific information need. In our system, users specify their need in the form of a natural language query, and are provided with recommended documents based on ratings by other users with similar questions. Preliminary experiments show that the collaborative filtering recommendations increase the efficiency of the document search process. We also discuss some key challenges of designing a collaborative filtering system for document search.

Recommender systems have been evaluated in many, often incomparable, ways. In this article, we review the key decisions in evaluating collaborative filtering recommender systems: the user tasks being evaluated, the types of analysis and datasets being used, the ways in which prediction quality is measured, the evaluation of prediction attributes other than quality, and the user-based evaluation of the system as a whole. In addition to reviewing the evaluation strategies used by prior researchers, we present empirical results from the analysis of various accuracy metrics on one content domain where all the tested metrics collapsed roughly into three equivalence classes. Metrics within each equivalency class were strongly correlated, while metrics from different equivalency classes were uncorrelated.

Current approaches to presentation of mathematics (on paper or in electronic format) have usability drawbacks that make learning and appreciation of mathematics challenging and often frustrating. We propose a development of a software user toolkit aimed to facilitate the creation of highly usable and effective presentations of mathematical ideas. In this paper we identify three problems which readers of math documents usually experience, and give a vision of how we might address those problems by designing a more interactive and intelligent interface

Automated collaborative filtering (ACF) systems predict a person's affinity for items or information by connecting that person's recorded interests with the recorded interests of a community of people and sharing ratings between like-minded persons. However, current recommender systems are black boxes, providing no transparency into the working of the recommendation. Explanations provide that transparency, exposing the reasoning and data behind a recommendation. In this paper, we address explanation interfaces for ACF systems - how they should be implemented and why they should be implemented. To explore how, we present a model for explanations based on the user's conceptual model of the recommendation process. We then present experimental results demonstrating what components of an explanation are the most compelling. To address why, we present experimental evidence that shows that providing explanations can improve the acceptance of ACF systems. We also describe some initial explorations into measuring how explanations can improve the filtering performance of users.

% & ' ( ) * !+ , -) ./ 0 1 32 -4 5 6 1 & ) * 7 98 ": <; '= ?> <; ": A@ B DC &E )F HG I8 KJ L 9E )F NM O7 "8 QP RE 6S T> AU B <V W8 ":

Collaborative filtering systems help address information overload by usingthe opinions of users in a community to make personal recommendations fordocuments to each user. Many collaborative filtering systems have few useropinions relative to the large number of documents available. This sparsityproblem can reduce the utility of the filtering system by reducing thenumber of documents for which the system can make recommendations andadversely affecting the quality of recommendations. This paper defines andimplements a model for integrating content-based ratings into acollaborative filtering system. The filterbot model allows collaborativefiltering systems to address sparsity by tapping the strength of contentfiltering techniques. We identify and evaluate metrics for assessing theeffectiveness of filterbots specifically, and filtering system enhancementsin general. Finally, we experimentally validate the filterbot approach byshowing that even simple filterbots such as spell checking can increase theutility for users of sparsely populated collaborative filtering systems.Keywords Collaborative filtering, information filtering, content analysis,recommendation systems, social filtering, GroupLens Research, informationfiltering agents.

No abstract available.