Practical constrains of user interfaces make the user's judgment (during the feedback loop) deviate from real thoughts (when the full document is read).This is often overlooked in evaluation of relevance feedback.This paper quantitatively analyze the impact of judging inconsistency on the performance of relevance feedback.

Abstract We introduce a framework of multiple viewpoint systems for describing and designing systems that use more than one representation or set of relevance judgments on the same collection. A viewpoint is any representational scheme on some collection of data objects together with a mechanism for accessing this content. A multiple viewpoint system allows a searcher to pose queries to one viewpoint and then change to another viewpoint while retaining a sense of context. Multiple viewpoint systems are well suited to alleviate vocabulary mismatches and to take advantage of the possibility of combining evidence. We discuss some of the issues that arise in designing and using such systems and illustrate the concepts with several examples.

The proliferation of online information resources increases the importance of effective and efficient information retrieval in a multicollection environment. Multicollection searching is cast in three parts: collection selection (also referred to as database selection), query processing and results merging. In this work, we focus our attention on the evaluation of the first step, collection selection.In this article, we present a detailed discussion of the methodology that we used to evaluate and compare collection selection approaches, covering both test environments and evaluation measures. We compare the CORI , CVV and gGLOSS collection selection approaches using six test environments utilizing three document testbeds. We note similar trends in performance among the collection selection approaches, but the CORI approach consistently outperforms the other approaches, suggesting that effective collection selection can be achieved using limited information about each collection.The contributions of this work are both the assembled evaluation methodology as well as the application of that methodology to compare collection selection approaches in a standardized environment.

Vocabulary incompatibilities arise when the terms used to index a document collection are largely unknown, or at least not well-known to the users who eventually search the collection. No matter how comprehensive or well-structured the indexing vocabulary, it is of little use if it is not used effectively in query formulation. This paper demonstrates that techniques for mapping user queries into the controlled indexing vocabulary have the potential to radically improve document retrieval performance. We also show how the use of controlled indexing vocabulary can be employed to achieve performance gains for collection selection. Finally, we demonstrate the potential benefit of combining these two techniques in an interactive retrieval environment. Given a user query, our evaluation approach simulates the human user's choice of terms for query augmentation given a list of controlled vocabulary terms suggested by a system. This strategy lets us evaluate interactive strategies without the need for human subjects.

Abstract The proliferation of online information resources increases the importance of effective and efficient distributed searching. Distributed searching is cast in three parts – database selection, query processing, and results merging. In this paper we examine the effect of database selection on retrieval performance. We look at retrieval performance in three different distributed retrieval testbeds and distill some general results. First we find that good database selection can result in better retrieval effectiveness than can be achieved in a centralized database. Second we find that good performance can be achieved when only a few sites are selected and that the performance generally increases as more sites are selected. Finally we find that when database selection is employed, it is not necessary to maintain collection wide information (CWI), e.g. global idf. Local information can be used to achieve superior performance. This means that distributed systems can be engineered with more autonomy and less cooperation. This work suggests that improvements in database selection can lead to broader improvements in retrieval performance, even in centralized (i.e. single database) systems. Given a centralized database and a good selection mechanism, retrieval performance can be improved by decomposing that database conceptually and employing a selection step. 1

As more online databases are integrated into digital libraries, the issue of quality control of the data becomes increasingly important, especially as it relates to the effective retrieval of information. Authority work, the need to discover and reconcile variant forms of strings in bibliographic entries, will become more critical in the future. Spelling variants, misspellings, and transliteration differences will all increase the difficulty of retrieving information. We investigate a number of approximate string matching techniques that have traditionally been used to help with this problem. We then introduce the notion of approximate word matching and show how it can be used to improve detection and categorization of variant forms. We demonstrate the utility of these approaches using data from the Astrophysics Data System and show how we can reduce the human effort involved in the creation of authority files.

Clustering partitions a collection of objects into groups called clusters, such that similar objects fall into the same group. Similarity between objects is defined by a distance function satisfying the triangle inequality; this distance function along with the collection of objects describes a distance space. In a distance space, the only operation possible on data objects is the computation of distance between them. All scalable algorithms in the literature assume a special type of distance space, namely a k-dimensional vector space, which allows vector operations on objects. We present two scalable algorithms designed for clustering very large datasets in distance spaces. Our first algorithm BUBBLE is, to our knowledge, the first scalable clustering algorithm for data in a distance space. Our second algorithm BUBBLE-FM improves upon BUBBLE by reducing the number of calls to the distance function, which may be computationally very expensive. Both algorithms make only a single scan over the database while producing high clustering quality. In a detailed experimental evaluation, we study both algorithms in terms of scalability and quality of clustering. We also show results of applying the algorithms to a real life dataset.

We compare the performance of two database selection algorithms reported in the literature.Their performance is compared using a common testbed designed specifically for database selection techniques.The testbed is a decomposition of the TREC/TIPSTER data into 236 subcollections.The databases from our testbed were ranked using both the gGlOSS and CORI techniques and compared to a baseline derived from TREC relevance judgements.We examined the degree to which CORI and gGlOSS approximate this baseline.Our results con rm our earlier observation that the gGlOSS Ideal(l) ranks do not estimate relevancebased ranks well.We also nd that CORI is a uniformly better estimator of relevance-based ranks than gGlOSS for the test environment used in this study.Part of the advantage of the CORI algorithm can be explained by a strong correlation between gGlOSS and a size-based baseline (SBR).We also nd that CORI produces consistently accurate rankings on testbeds ranging from 100{921 sites.However for a given level of recall, search eort appears to scale linearly with the number of databases.

We describe a testbed for database selection techniques and an experiment conducted using this testbed.The testbed is a decomposition of the TREC/TIPSTER data that allows analysis of the data along multiple dimensions, including collection-based and temporal-based analysis.We characterize the subcollections in this testbed in terms of number of documents, queries against which the document,s have been evaluated for relevance, and distribution of relevant documents.We then present initial results from a study conducted using this testbed that examines the effectiveness of the gGlOSS approach to database selection.The databases from our testbed were ranked using the gGl0S.S techniques and compared to the gGlOSS I&l(l) baseline and a baseline derived from TREC relevance judgements.We have examined the degree to which several gGlOSS estimate functions approximate these baselines.Our initial results confirm that the gGZOSS estimators are excellent predictors of the Ideal(Z) ranks but that the Ideal(l) ranks do not estimate relevance-based ranks well.

Large scientific applications which rely on highly parallel computational analysis require highly parallel data access. We describe an object-oriented scientific database system that achieves nearly linear scale-up over large, million object data sets. Of primary importance are those features which seem central to the development of this, or any other parallel database system. These include techniques of object distribution, of multi-operator parallelism, and of indirect object referencing. It also appears to require a query server architecture instead of the more common page server configurations.

As more online databases are integrated into digital libraries, the issue of quality control of the data becomes increasingly important, especially as it relates to the effective retrieval of information. The need to discover and reconcile variant forms of strings in bibliographic entries, i.e., authority work, will become more difficult. Spelling variants, misspellings, and transliteration differences will all increase the difficulty of retrieving information. Approximate string matching has traditionally been used to help with this problem. In this paper we introduce the notion of approximate word matching and show how it can be used to improve detection and categorization of variant forms. 1 Introduction There are increasingly more online databases in the current climate of electronic publishing. The challenge is to integrate them into coherent digital libraries that let users have unimpeded access to accurate information. As the pace of electronic publication accelerates there wil...

One of the difficulties in using current Boolean-based information retrieval systems is that it is hard for a user, especially a novice, to formulate an effective Boolean query. Query reformulation can be even more difficult and complex than formulation since users often have difficulty incorporating the new information gained from the previous search into the next query. In this article, query reformulation is viewed as a classification problem, that is, classifying documents as either relevant or nonrelevant. A new reformulation algorithm is proposed which builds a tree-structured classifier, called a query tree, at each reformulation from a set of feedback documents retrieved from the previous search. The query tree can easily be transformed into a Boolean query. The query tree is compared to two query reformulation algorithms on benchmark test sets (CACM, CISI, and Medlars). In most experiments, the query tree showed significant improvements in precision over the two algorithms compared in this study. We attribute this improved performance to the ability of the query tree algorithm to select good search terms and to represent the relationships among search terms into a tree structure. © 1997 John Wiley & Sons, Inc.

Using the vector space information retrieval model, we show that the update of term weights under document insertions is computationally expensive for weighting schemes that use collection statistics and normalization by document vector lengths. In the dynamic setting, we argue that strict adherence to such schemes is impractical and unnecessary as long as retrieval effectiveness commensurate with strict adherence is attained. Experiments using standard test collections as a source of document insertions support this argument. These experiments indicate that term weights may drift from their mathematically defined values without a serious loss of retrieval effectiveness. The only problematic setting is when new terms are present in newly inserted documents. Ignoring these terms can cause an effectiveness degradation. 1 Introduction The rapid growth in online information has fueled recent interest in techniques to handle the burgeoning flood of data becoming electronically available. ...

We find that dissemination of collection wide information (CWI) in a distributed collection of documents is needed to achieve retrieval effectiveness comparable to a centralized collection.Complete dissemination is unnecessary.The required dissemination level depends upon how documents are allocated among sites.Low dissemination is needed for random document allocation, but higher levels are needed when documents are allocated based on content.We define parameters to control dissemination and document allocation and present results from four test collections.We define the notion of iso-knowledge lines with respect to the number of sites and level of dissemination in the distributed archive, and show empirically that iso-knowledge lines are also zsoefiectivene.wlines when documents are randomly allocated.

Scientific applications often manipulate very large sets of persistent data. Over the past decade, advances in disk storage device performance have consistently been outpaced by advances in the performance of the rest of the computer system. As a result, many scientific applications have become I/O-bound, i.e. their run-times are dominated by the time spent performing I/O operations. Consequently, the performance of I/O operations has become critical for high performance in these applications. The ELFS approach is designed to address the issue of high performance I/O by treating files as typed objects. Typed file objects can exploit knowledge about the file structure and type of data. Typed file objects can selectively apply techniques such as prefetching, parallel asynchronous file access, and caching to improve performance. Also, by typing objects, the interface to the user can be improved in two ways. First, the interface can be made easier to use by presenting file operations in a more natural manner to the user. Second, the interface can allow the user to provide an “oracle” about access patterns, that can allow the file object to improve performance. By combining these concepts with the object-oriented paradigm, the goal of the ELFS methodology is to create flexible, extensible file classes that are easy to use while achieving high performance. In this paper we present the ELFS approach and our experiences with the design and implementation of two file classes: a two dimensional dense matrix file class and a multidimensional range searching file class.

As CPU performance has rapidly improved, increased pressure has been placed on the performance of accessing external data in order to keep up with demand. Increasingly often the I/O subsystem and related software is unable to meet this demand and valuable CPU resources are left underutilized, while users are forced to wait longer than necessary for results. This problem is especially acute for many scientific applications which use large data sets and require high performance. The paper presents experiences with working to alleviate an I/O bottleneck in radio astronomy applications at the National Radio Astronomy Observatory (NRAO). It presents the model, the ExtensibLe File Systems model (ELFS), for improving both the performance of data access and the usability of access software. It then presents the current situation at NRAO, the solution, performance results and plans for future work.>

The Intel Concurrent File System (CFS) for the iPSC/2 hypercube is one of the first production file systems to utilize the declustering of large files across numbers of disks to improve I/O performance. The CFS also makes use of dedicated I/O nodes, operating asynchronously, which provide file caching and prefetching. Processing of I/O requests is distributed between the compute node that initiates the request and the I/O nodes that service the request. The effects of the various design decisions in the Intel CFS are difficult to determine without measurements of an actual system. We present performance measurements of the CFS for a hypercube with 32 compute nodes and four I/0 nodes (four disks). Measurement of read/write rates for one compute node to one I/O node, one compute node to multiple I/O nodes, and multiple compute nodes to multiple I/O nodes form the basis for the study. Additional measurements show the effects of different buffer sizes, caching, prefetching, and file preallocation on system performance.