Knowledge workers must manage large numbers of simultaneous, ongoing projects that collectively involve huge numbers of resources (documents, emails, web pages, calendar items, etc). An activity database that captures the relationships among projects, resources, and time can drive a variety of tools that save time and increase productivity. To maximize net time savings, we would prefer to build such a database automatically, or with as little user effort as possible. In this paper, we present several sets of features and algorithms for predicting the project associated with each action a user performs on the desktop. Key to our methods is salience, the notion that more recent activity is more informative. By developing novel features that represent salience, we were able to learn models that outperform both a simple benchmark and an expert system tuned specifically for this task on real-world data from five users.

This paper focuses on human-machine communication with intelligent agents, it proposes a generic architecture with an algorithm for natural language (NL) command interpretation which makes it easy to define different applications using the descriptio

Email-based activity management systems promise to give users better tools for managing increasing volumes of email, by organizing email according to a user's activities. Current activity management systems do not automatically classify incoming messages by the activity to which they belong, instead relying on simple heuristics (such as message threads), or asking the user to manually classify incoming messages as belonging to an activity. This paper presents several algorithms for automatically recognizing emails as part of an ongoing activity. Our baseline methods are the use of message reply-to threads to determine activity membership and a naïve Bayes classifier. Our SimSubset and SimOverlap algorithms compare the people involved in an activity against the recipients of each incoming message. Our SimContent algorithm uses IRR (a variant of latent semantic indexing) to classify emails into activities using similarity based on message contents. An empirical evaluation shows that each of these methods provide a significant improvement to the baseline methods. In addition, we show that a combined approach that votes the predictions of the individual methods performs better than each individual method alone.

Abstract Current document‐retrieval tools succeed in locating large numbers of documents relevant to a given query. While search results may be relevant according to the topic of the documents, it is more difficult to identify which of the relevant documents are most suitable for a particular user. Automatic genre analysis (i.e., the ability to distinguish documents according to style) would be a useful tool for identifying documents that are most suitable for a particular user. We investigate the use of machine learning for automatic genre classification. We introduce the idea of domain transfer—genre classifiers should be reusable across multiple topics—which does not arise in standard text classification. We investigate different features for building genre classifiers and their ability to transfer across multiple‐topic domains. We also show how different feature‐sets can be used in conjunction with each other to improve performance and reduce the number of documents that need to be labeled.

Comparative evaluation of Machine Learning (ML) systems used for Information Extraction (IE) has suffered from various inconsistencies in experimental procedures. This paper reports on the results of the Pascal Challenge on Evaluating Machine Learning for Information Extraction, which provides a standardised corpus, set of tasks, and evaluation methodology. The challenge is described and the systems submitted by the ten participants are briefly introduced and their performance is analysed.

We propose a novel ensemble learning algorithm called Triskel, which has two interesting features. First, Triskel learns an ensemble of classifiers, each biased to have high precision on instances from a single class (as opposed to, for example, boosting, where the ensemble members are biased to maximise accuracy over a subset of instances from all classes). Second, the ensemble members' voting weights are assigned so that certain pairs of biased classifiers outweigh the rest of the ensemble, if their predictions agree. Our experiments demonstrate that Triskel often outperforms boosting, in terms of both accuracy and training time. We also present an ROC analysis, which shows that Triskel's iterative structure corresponds to a sequence of nested ROC spaces. The analysis predicts that Triskel works best when there are concavities in the ROC curves; this prediction agrees with our empirical results.

Many structured activities are managed by email. For instance, a consumer purchasing an item from an e-commerce vendor may receive a message confirming the order, a warning of a delay, and then a shipment notification. Existing email clients do not understand this structure, forcing users to manage their activities by sifting through lists of messages. As a first step to developing email applications that provide high-level support for structured activities, we consider the problem of automatically learning an activity's structure. We formalize activities as finite-state automata, where states correspond to the status of the process, and transitions represent messages sent between participants. We propose several unsupervised machine learning algorithms in this context, and evaluate them on a collection of e-commerce email.

Collaborative recommendation has emerged as an effective technique for personalized information access. However, there has been relatively little theoretical analysis of the conditions under which the technique is effective. To explore this issue, we analyse the robustness of collaborative recommendation: the ability to make recommendations despite (possibly intentional) noisy product ratings. There are two aspects to robustness: recommendation accuracy and stability. We formalize recommendation accuracy in machine learning terms and develop theoretically justified models of accuracy. In addition, we present a framework to examine recommendation stability in the context of a widely-used collaborative filtering algorithm. For each case, we evaluate our analysis using several real-world data-sets. Our investigation is both practically relevant for enterprises wondering whether collaborative recommendation leaves their marketing operations open to attack, and theoretically interesting for the light it sheds on a comprehensive theory of collaborative recommendation.

Distributed heterogeneous search systems are an emerging phenomenon in Web search, in which independent topic-specific search engines provide search services, and metasearchers distribute user's queries to only the most suitable search engines. Previous research has investigated methods for engine selection and merging of search results (i.e. performance improvements from the user's perspective) . We focus instead on performance from the service provider's point of view (e.g, income from queries processed vs. resources used to answer them). We consider a scenario in which individual search engines compete for user queries by choosing which documents (topics) to index. The difficulty here stems from the fact that the utilities of local engine actions should depend on the uncertain actions of competitors. Thus, naive strategies (e.g, blindly indexing lots of popular documents) are ineffective. We model the competition between search engines as a stochastic game, and propose a reinforcement learning approach to managing search index contents. We evaluate our approach using a large log of user queries to 47 real search engines.

Distributed heterogeneous search environments are an emerging phenomenon in Web search. We provide a theoretical analysis of the problem and propose a method, utilising reinforcement learning techniques, for automatically managing search engine content. We examine the problem of performance-maximising behaviour for noncooperative specialised search engines in heterogeneous search environments. In particular, we analyse a scenario in which individual search engines compete for queries by choosing which documents to index. We provide game-theoretic analysis of a simplified version of the problem and motivate the use of the concept of "bounded rationality". We then cast our problem as a reinforcement learning task, where the goal of a specialised search engine is to exploit suboptimal behaviour of its competitors to improve own performance.

Abstract Several languages for querying and transforming XML, including XML‐QL, Quilt, and XQL, have been proposed. However, these languages do not support ranked queries based on textual similarity, in the spirit of traditional IR. Several extensions to these XML query languages to support keyword search have been made, but the resulting languages cannot express IR‐style queries such as “find books and CDs with similar titles.” In some of these languages keywords are used merely as boolean filters without support for true ranked retrieval; others permit similarity calculations only between a data value and a constant, and thus cannot express the above query. WHIRL avoids both problems, but assumes relational data. We propose ELIXIR, an expressive and efficient language for XML information retrieval that extends XML‐QL with a textual similarity operator that can be used for similarity joins, so ELIXIR is sufficiently expressive to handle the sample query above. ELIXIR thus qualifies as a general‐purpose XML IR query language. Our central contribution is an efficient algorithm for answering ELIXIR queries that rewrites the original ELIXIR query into a series of XML‐QL queries to generate intermediate relational data, and uses WHIRL to efficiently evaluate the similarity operators on this intermediate data, yielding an XML document with nodes ranked by similarity. Our experiments demonstrate that our prototype scales well with the size of the query and the XML data.

The emergence of XML as a standard interchange format for structured documents/data has given rise to many XML query language proposals. However, some of these languages do not support information retrieval-style ranked queries based on textual similarity. There have been several extensions to these query languages to support keyword search, but the resulting query languages cannot express queries such as``find books and CDs with similar titles''. Either these extensions use keywords as mere boolean filters, or similarities can be calculated only between data values and constants rather than two data values. We propose ELIXIR, an \textbf{\underline{e}}xpressive and \textbf{\underline{e}}fficient\textbf{\underline{l}}anguage for \textbf{\underline{X}}ML \textbf{\underline{i}}nformation \textbf{\underline{r}}etrieval that extends the query language XML-QL \cite{deutsch-www8,deutsch-deb99} with a textual similarity operator. ELIXIR is a general-purpose XML information retrieval language, sufficiently expressive to handle the above query. Our algorithm for answering ELIXIR queries rewrites the original ELIXIR query into a series of XML-QL queries that generate intermediate relational data, and uses relational database techniques to efficiently evaluate the similarity operators on this intermediate data, yielding an XML document with nodes ranked by similarity. Our experiments demonstrate that our prototype scales well with the size of the XML data and complexity of the query.