A thesaurus contains a set of terms or features that may be used to represent recorded information, including prose documents or scientific data sets. The focus of this work is on the basic structural nature of a thesaurus itself, not on how people develop a thesaurus or how a thesaurus effects retrieval performance. Thesauri in this research are automatically developed in a simulation from sets of randomly or exhaustively generated documents. Each thesaurus is generated by the Thesaurus Generator software from a set of several hundred documents, and thousands of different document sets are used as input to the Thesaurus Generator, producing thousands of thesauri. Thus, thousands of thesauri are generated for each data point in accompanying graphs. The characteristics of this large number of thesauri are studied so that the relationships between thesaurus parameters can be determined. Some rules governing these relationships are suggested, addressing factors such as tree height and width, number of tree roots in thesauri, and number of terms available for the vocabulary. How these parameters scale as vocabularies grow is addressed. These results apply to various information systems that contain features with hierarchical relationships, including many thesauri and ontologies.

The assignment of indexing terms and metadata to documents, data, and other information representations is considered useful, but the utility of including a single term is seldom discussed. The author discusses a simple model of document ordering and then shows how assigning index and metadata labels improves or decreases retrieval performance. The Indexing and Metadata Advantage (IMA) factor measures how indexing or assigning a metadata term helps (or hurts) ordering performance. Performance values and the associated IMA expressions are computed, consistent with several different assumptions. The economic value associated with various term assignment decisions is developed. The IMA term advantage model itself is empirically validated with computer software that shows that the analytic results obtained agree completely with the actual performance gains and losses found when ordering all sets of 14 or fewer documents. When the formulas in the software are changed to differ from this model, the predictions of the actual performance are erroneous.

Relationships between terms and features are an essential component of thesauri, ontologies, and a range of controlled vocabularies. In this article, we describe ways to identify important concepts in documents using the relationships in a thesaurus or other vocabulary structures. We introduce a methodology for the analysis and modeling of the indexing process based on a weighted random walk algorithm. The primary goal of this research is the analysis of the contribution of thesaurus structure to the indexing process. The resulting models are evaluated in the context of automatic subject indexing using four collections of documents pre‐indexed with 4 different thesauri (AGROVOC [UN Food and Agriculture Organization], high‐energy physics taxonomy [HEP], National Agricultural Library Thesaurus [NALT], and medical subject headings [MeSH]). We also introduce a thesaurus‐centric matching algorithm intended to improve the quality of candidate concepts. In all cases, the weighted random walk improves automatic indexing performance over matching alone with an increase in average precision (AP) of 9% for HEP, 11% for MeSH, 35% for NALT, and 37% for AGROVOC. The results of the analysis support our hypothesis that subject indexing is in part a browsing process, and that using the vocabulary and its structure in a thesaurus contributes to the indexing process. The amount that the vocabulary structure contributes was found to differ among the 4 thesauri, possibly due to the vocabulary used in the corresponding thesauri and the structural relationships between the terms. Each of the thesauri and the manual indexing associated with it is characterized using the methods developed here.

Abstract When do information retrieval systems using two document clusters provide better retrieval performance than systems using no clustering? We answer this question for one set of assumptions and suggest how this may be studied with other assumptions. The “Cluster Hypothesis” asks an empirical question about the relationships between documents and user‐supplied relevance judgments, while the “Cluster Performance Question” proposed here focuses on the when and why of information retrieval or digital library performance for clustered and unclustered text databases. This may be generalized to study the relative performance of m versus n clusters.

The major emphasis is on analytical techniques for predicting the performance of various collection fusion scenarios. Knowledge of analytical models of information retrieval system performance, both with single processors and with multiple processors, increases our understanding of the parameters (e.g., number of documents, ranking algorithms, stemming algorithms, stop word lists, etc.) affecting system behavior. While there is a growing literature on the implementation of distributed information retrieval systems and digital libraries, little research has focused on analytic models of performance. We analytically describe the performance for single and multiple processors, both when different processors have the same parameter values and when they have different values. The use of different ranking algorithms and parameter values at different sites is examined.

Abstract There are regularities in the statistical information provided by natural language terms about neighboring terms. We find that when phrase rank increases, moving from common to less common phrases, the value of the expected mutual information measure (EMIM) between the terms regularly decreases. Luhn's model suggests that midrange terms are the best index terms and relevance discriminators. We suggest reasons for this principle based on the empirical relationships shown here between the rank of terms within phrases and the average mutual information between terms, which we refer to as the Inverse Representation—EMIM principle. We also suggest an Inverse EMIM term weight for indexing or retrieval applications that is consistent with Luhn's distribution. An information theoretic interpretation of Zipf's Law is provided. Using the regularity noted here, we suggest that Zipf's Law is a consequence of the statistical dependencies that exist between terms, described here using information theoretic concepts.

The variety of performance measures available for information retrieval systems, search engines, and network filtering agents can be confusing to both practitioners and scholars. Most discussions about these measures address their theoretical foundations and the characteristics of a measure that make it desirable for a particular application. In this work, we consider how measures of performance at a point in a search may be formally compared. Criteria are developed that allow one to determine the percent of time or conditions under which two different performance measures suggest that one document ordering is superior to another ordering, or when the two measures disagree about the relative value of document orderings. As an example, graphs provide illustrations of the relationships between precision and F.

Instead of using traditional performance measures such as precision and recall, information retrieval performance may be measured by considering the probability that the search engine is optimal and the difficulty associated with retrieving documents with a given query or on a given topic. These measures of desirable characteristics are more easily and more directly interpretable than are traditional measures. The performance of the Target and Freestyle search engines is examined, and is very good. Each query in the CF database is assigned a difficulty number, and these numbers are found to strongly correlate with other measures of retrieval performance such as an E or F value. The query difficulty correlates weakly with query length.

Whether using Boolean queries or ranking documents using document and term weights will result in better retrieval performance has been the subject of considerable discussion among document retrieval system users and researchers. We suggest a method that allows one to analytically compare the two approaches to retrieval and examine their relative merits. The performance of information retrieval systems may be determined either by using experimental simulation, or through the application of analytic techniques that directly estimate the retrieval performance, given values for query and database characteristics. Using these performance predicting techniques, sample performance figures are provided for queries using the Boolean and or, as well as for probabilistic systems assuming statistical term independence or term dependence. The variation of performance across sublanguages (used in different academic disciplines) and queries is examined. The performance of models failing to meet statistical and other assumptions is examined. © 1997 John Wiley & Sons, Inc.

Information may be defined as the characteristics of the output of a process, these being informative about the process and the input. This discipline independent definition may be applied to all domains, from physics to epistemology. Hierarchies of processes, linked together, provide a communication channel between each of the corresponding functions and layers in the hierarchies. Models of communication (Shannon), perception, observation, belief, and knowledge are suggested that are consistent with this conceptual framework of information as the value of the output of any process in a hierarchy of processes. Misinformation and errors are considered. © 1997 John Wiley & Sons, Inc.

An analytic method of information retrieval and filtering evaluation can quantitatively predict the expected number of documents examined in retrieving a relevant document. It also allows researchers and practitioners to qualitatively understand how varying different estimates of query parameter values affects retrieval performance. The incorporation of relevance feedback to increase our knowledge about the parameters of relevant documents and the robustness of parameter estimates is modeled. Single term and two term independence models, as well as a complete term dependence model, are developed. An economic model of retrieval performance may be used to study the effects of database size and to provide analytic answers to questions comparing retrieval from small and large databases, as well as questions about the number of terms in a query. Results are presented as a performance surface, a three dimensional graph showing the effects of two independent variables on performance. © 1996 John Wiley & Sons, Inc.

The use of terms from natural and social scientific titles and abstracts is studied from the perspective of sublanguages and their specialized dictionaries. Different notions of sublanguage distinctiveness are explored. Objective methods for separating hard and soft sciences are suggested based on measures of sublanguage use, dictionary characteristics, and sublanguage distinctiveness. Abstracts were automatically classified with a high degree of accuracy by using a formula that considers the degree of uniqueness of terms in each sublanguage. This may prove useful for text filtering or information retrieval systems. © 1995 John Wiley & Sons, Inc.

A document classifier places documents together in a linear arrangement for browsing or high-speed access by human or computerized information retrieval systems. Requirements for document classification and browsing systems are developed from similarity measures, distance measures, and the notion of subject aboutness. A requirement that documents be arranged in decreasing order of similarity as the distance from a given document increases can often not be met. Based on these requirements, information-theoretic considerations, and the Gray code, a classification system is proposed that can classify documents without human intervention. It provides a theoretical justification for individual classification numbers going from broad to narrow topics when moving from left to right in the classification number. A general measure of classifier performance is developed, and used to evaluate experimental results comparing the distance between subject headings assigned to documents given classifications from the proposed system and the Library of Congress Classification (LCC) system. Browsing in libraries, hypertext, and databases is usually considered to be the domain of subject searches. The proposed system can incorporate both classification by subject and by other forms of bibliographic information, allowing for the generalization of browsing to include all features of an information carrying unit. © 1992 John Wiley & Sons, Inc.

The performance of medical information retrieval systems is measured using historical data or predicted using formal probabilistic methods derived from artificial intelligence and statistical decision theoretic considerations. Technique have been described that assist the searcher with a query or information need by providing graphs showing the quality of past retrieval performance for that specific query, as well as expected future performance. Documents or text fragments (from a hypertext system) are ranked for possible presentation to the searcher based on the document of fragment's odds of relevance. The expected performance is computed from knowledge gained from relevance judgments provided by the searcher about the quality of the retrieved documents, as well as any system knowledge available about possible initial values of parameters of distributions describing the occurrence of features of relevance and all text fragments. The individual documents do not need to be examined to predict performance.>

A collection of data, whether encoded as a relational database or as a document collection, contains information that may be measured. We assume that a maximally informative collection is desirable; that is, given the constraints of capacity and costs of acquisition and storage, the best data collection is the collection containing the greatest amount of information. Several measures of information are described and applied to the measurement of the information content of relations, documents, and books, both singly and in collections. The characteristics of the measures are examined, including the ability of a measure to predict the amount of information that will be available. The utility theoretic measure proposed by Belis is argued to be the most appropriate measure of collection information. It is used to compare the information in two library collections. The methods of counting database or document uses, or treating library circulations as a measure of collection quality, are seen to be consistent with the Belis information measure. © 1990 John Wiley & Sons, Inc.

A probabilistic document-retrieval system may be seen as a sequential learning process, in which the system learns the characteristics of relevant documents, or more formally, it learns the parameters of probability distributions describing the frequencies of feature occurrences in relevant and nonrelevant documents. Probability distributions that may be used to describe the distribution of features include binary and Poisson distributions. Techniques for estimating the parameters of distributions are suggested. We have tested a proposal that parameters of distributions describing the distribution of features in nonrelevant documents be estimated from the parameters of the corresponding distributions of the entire database; the confidence parameter of such an estimate resulting in the highest average precision is given. Tests of several methods for estimating the parameters of distributions describing the distribution of features in relevant documents suggest that small values of the confidence parameter be used in our initial estimates of parameters for relevant documents. © 1988 John Wiley & Sons, Inc.

Probabilistic models of document-retrieval systems incorporating sequential learning through relevance feedback may require frequent and time-consuming reevaluations of documents. Coordination level matching is shown to provide equivalent document rankings to binary models when term discrimination values are equal for all terms; this condition may be found, for example, in probabilistic systems with no feedback. A nearest-neighbor algorithm is presented which allows probabilistic sequential models consistent with two-Poisson or binary-independence assumptions to easily locate the “best” document using temporary sets of documents at a given coordination level. Conditions under which reranking is unnecessary are given. © 1987 John Wiley & Sons, Inc.

Probabilistic document retrieval systems consistent with the two Poisson independence model outperforms the binary independence model if the terms are distributed as described by the model's assumptions. The Two Poisson Effectiveness Hypothesis suggests that retrieval models based upon the two Poisson model will outperform binary independent models when used on a “real-world” database, where independence and two Poisson term occurrence distributions fail to hold, because the added information obtained from incorporating term frequency information will more than compensate for the non-Poisson distributions of terms. Searches of the MED1033 database suggest that if terms are not independent and frequencies of term occurrence are not distributed in a two Poisson manner, the binary independence sequential retrieval model outperforms the two Poisson independence retrieval model.