The literature on assigned indexing considers three possible viewpoints—the author's viewpoint as evidenced in the title, the users' viewpoint, and the indexer's viewpoint—and asks whether and which of those views should be reflected in an indexer's choice of terms to assign to an item. We study this question empirically, as opposed to normatively. Based on the literature that discusses whose viewpoints should be reflected, we construct a research model that includes those same three viewpoints as factors that might be influencing term assignment in actual practice. In the unique study design that we employ, the records of term assignments made by identified indexers in academic libraries are cross‐referenced with the results of a survey that those same indexers completed on political views. Our results indicate that in our setting, variance in term assignment was best explained by indexers' personal political views. © 2019 ASIS&T

In many social tagging systems, users can see the tags already created by others. Prior research has shown that this exposure leads users to create tags that are semantically related to the previous ones. We investigate two possible mechanisms through which this occurs, semantic priming and strategic choice. Semantic priming occurs when an existing tag subconsciously primes the user’s mind to suggest semantically related tags. In an experiment, no such effect is found, in contrast to prior research. A follow-up study shows that whether semantic priming occurs depends on whether the person uses others’ previously created tags or is just passively exposed to them. The second type of influence, strategic choice, occurs in ESP-type settings. It refers to behavior in which a user chooses words that are semantically related to an existing tag in order to increase the chances of matching one’s partner. Experimental results provide clear evidence of this strategic influence. In a follow-up study, we demonstrate that there is a meaningful difference in the tag sets that are created under the influence of strategic choice. Our work sheds light on the conditions and mechanisms through which existing tags influence subsequent tagging behavior.

In the context of online question‐answering services, an intermediary clarifies the user's needs by eliciting additional information. This research proposes that these elicitations will depend on the type of question. In particular, this research explores the relationship between three constructs: question types, elicitations, and the fee that is paid for the answer. These relationships are explored for a few different question typologies, including a new kind of question type that we call Identity. It is found that the kinds of clarifications that intermediaries elicit depend on the type of question in systematic ways. A practical implication is that interactive question‐answering services—whether human or automated—can be steered to focus attention on the kinds of clarification that are evidently most needed for that question type. Further, it is found that certain question types, as well as the number of elicitations, are associated with higher fees. This means that it may be possible to define a pricing structure for question‐answering services based on objective and predictable characteristics of the question, which would help to establish a rational market for this type of information service. The newly introduced Identity question type was found to be especially reliable in predicting elicitations and fees.

Abstract The user‐centered approach to information retrieval emphasizes the importance of a user model in determining what information will be most useful to a particular user, given their context. Mediated search provides an opportunity to elaborate on this idea, as an intermediary's elicitations reveal what aspects of the user model they think are worth inquiring about. However, empirical evidence is divided over whether intermediaries actually work to develop a broadly conceived user model. Our research revisits the issue in a web research services setting, whose characteristics are expected to result in more thorough user modeling on the part of intermediaries. Our empirical study confirms that intermediaries engage in rich user modeling. While intermediaries behave differently across settings, our interpretation is that the underlying user model characteristics that intermediaries inquire about in our setting are applicable to other settings as well.

Abstract In the traditional model of information retrieval, searchers and indexers choose query and index terms, respectively, and these term choices are ultimately compared in a matching process. One of the main challenges in information science and information retrieval is that searchers and indexers often do not choose the same term even though the item is relevant to the need whereas at other times they do choose the same term even though it is not relevant. But if both searchers and indexers have the opportunity to review feedback data showing the success or failure of their previous term choices, then there exists an evolutionary force that, all else being equal, will lead to helpful convergence in searchers' and indexers' term usage when the information is relevant, and helpfuldivergence of term usage when it is not. Based on learning theory, and new theory presented here, it is possible to predict which terms will emerge as the terminological conventions that are used by groups of searchers and the indexers of relevant and nonrelevant information items.

All information exchange on the Internet -- whether through full text, controlled vocabularies, ontologies, or other mechanisms -- ultimately requires that that an information provider and seeker use the same word or symbol. In this paper, we investigate what happens when both searchers and authors are dynamically choosing terms to match the other side. With each side trying to anticipate the other, does a terminological convention ever emerge, or do searchers and providers continue to miss potential partners through mis-match of terms? We use a game-theoretic setup to frame questions, and learning theory to make predictions about whether and which term will emerge as a convention.

How good is an IR test collection? A series of papers in recent years has addressed the question by empirically enumerating the consistency of performance comparisons using alternate subsets of the collection. In this paper we propose using Test Theory, which is based on analysis of variance and is specifically designed to assess test collections. Using the method, we not only can measure test reliability after the fact, but we can estimate the test collection's reliability before it is even built or used. We can also determine an optimal allocation of resources before the fact, e.g. whether to invest in more judges or queries. The method, which is in widespread use in the field of educational testing, complements data-driven approaches to assessing test collections. Whereas the data-driven method focuses on test results, test theory focuses on test designs. It offers unique practical results, as well as insights about the variety and implications of alternative test designs.

Abstract The concept of relevance has received a great deal of theoretical attention. Separately, the relationship between focused search and browsing has also received extensive theoretical attention. This article aims to integrate these two literatures with a model and an empirical study that relate relevance in focused searching to relevance in browsing. Some factors affect both kinds of relevance in the same direction; others affect them in different ways. In our empirical study, we find that the latter factors dominate, so that there is actually a negative correlation between the probability of a document's relevance to a browsing user and its probability of relevance to a focused searcher.

Abstract The view of documents and/or queries as random variables is gaining importance in the theory of information retrieval. We argue that traditional probabilistic models consider documents and queries as random variables, but that newer models such as language modeling and our unified model take this one step further. The additional step is called error in predictors. Such models consider that we don't observe the document and query random variables that are modeled to predict relevance probabilistically. Rather, there are additional random variables, which are the observed documents and queries. We discuss some important implications of this idea for parameter estimation, relevance prediction, and even test‐collection construction. By clarifying the positions of various probabilistic models on this question, and presenting in one place many of its implications, this article aims to deepen our common understanding of the theories behind traditional probabilistic models, and to strengthen the theoretical basis for further development of more recent approaches such as language modeling.

In this paper, we apply methods from educational testing to measure the reliability of an IR collection.

Abstract This paper proposes a new unified probabilistic model. Two previous models, Robertson et al.'s “Model 0” and “Model 3”, each have strengths and weaknesses. The strength of Model 0 not found in Model 3, is that it does not require relevance data about the particular document or query, and, related to that, its probability estimates are straightforward. The strength of Model 3 not found in Model 0 is that it can utilize feedback information about the particular document and query in question. In this paper we introduce a new unified probabilistic model that combines these strengths: the expression of its probabilities is straightforward, it does not require that data must be available for the particular document or query in question, but it can utilize such specific data if it is available. The model is one way to resolve the difficulty of combining two marginal views in probabilistic retrieval.

A central idea of Language Models is that documents (and perhaps queries) are random variables, generated by data-generating functions that are characterized by document (query) parameters. The key new idea of this paper is to model that a relevance judgment is also generated stochastically, and that its data generating function is also governed by those same document and query parameters. The result of this addition is that any available relevance judgments are easily incorporated as additional evidence about the true document and query model parameters. An additional aspect of this approach is that it also resolves the long-standing problem of document-oriented versus query-oriented probabilities. The general approach can be used with a wide variety of hypothesized distributions for documents, queries, and relevance. We test the approach on Reuters Corpus Volume 1, using one set of possible distributions. Experimental results show that the approach does succeed in incorporating relevance data to improve estimates of both document and query parameters, but on this data and for the specific distributions we hypothesized, performance was no better than two separate one-sided models. We conclude that the model's theoretical contribution is its integration of relevance models, document models, and query models, and that the potential for additional performance improvement over one-sided methods requires refinements.

Abstract We present a preliminary empirical test of a maximum likelihood approach to using relevance data for training information retrieval (IR) parameters. Similar to language models, our method uses explicitly hypothesized distributions for documents and queries, but we add to this an explicitly hypothesized distribution for relevance judgments. The method unifies document‐oriented and query‐oriented views. Performance is better than the Rocchio heuristic for document and/or query modification. The maximum likelihood methodology also motivates a heuristic estimate of the MLE optimization. The method can be used to test competing hypotheses regarding the processes of authors' term selection, searchers' term selection, and assessors' relevancy judgments.

We study an explicit parametric model of documents, queries, and rel- evancy assessment for Information Retrieval (IR). Mean-field methods are applied to analyze the model and derive efficient practical algorithms to estimate the parameters in the problem. The hyperparameters are es- timated by a fast approximate leave-one-out cross-validation procedure based on the cavity method. The algorithm is further evaluated on several benchmark databases by comparing with standard algorithms in IR.

Abstract Empirical work shows significant benefits from using relevance feedback data to improve information retrieval (IR) performance. Still, one fundamental difficulty has limited the ability to fully exploit this valuable data. The problem is that it is not clear whether the relevance feedback data should be used to train the system about what the users really mean, or about what the documents really mean. In this paper, we resolve the question using a maximum likelihood framework. We show how all the available data can be used to simultaneously estimate both documents and queries in proportions that are optimal in a maximum likelihood sense. The resulting algorithm is directly applicable to many approaches to IR, and the unified framework can help explain previously reported results as well as guide the search for new methods that utilize feedback data in IR.

Two competing approaches for document retrieval were first identified by Robertson, Maron, & Cooper (1982) for probabilistic retrieval. The difficulty of unifying those approaches was introduced as a problem of resolving query-focused with document-focused retrieval, and an approach towards unification was offered. That approach rests on a re-conceptualization of the meaning of term weight estimates. In this work, we propose a new unified model. The unification problem is re-framed as resulting from a lack of theory regarding the relationship of two sorts of data, absolute and relative. This new unified model is valid even for traditional interpretations of term estimates.

The introduction of computerized post-coordination has solved many of the problems of pre-coordinated subject access. However, the adoption of computerized post-coordination results in the loss of some pre-coordination benefits. Specifically, the effect of hiding terms within the context of others is lost in post-coordination which gives lead status to every document term. This results in spurious matches of terms out of context. Library patrons and Internet searchers are increasingly dissatisfied with subject access performance, in part because of unmanageably large retrieval sets. The need to enhance precision and limit the size of retrieval sets motivates this work which proposes partial coordination, an approach which incorporates the advantages of computer search with the ability of pre-coordination to limit spurious partial matches and thereby enhance precision.

Partial coordination is a new method for cataloging documents for subject access. It is especially designed to enhance the precision of document searches in online environments. This article reports a preliminary evaluation of partial coordination that shows promising results compared with full-text retrieval. We also report the difficulties in empirically evaluating the effectiveness of automatic full-text retrieval in contrast to mixed methods such as partial coordination which combine human cataloging with computerized retrieval. Based on our study, we propose research in this area will substantially benefit from a common framework for failure analysis and a common data set. This will allow information retrieval researchers adapting “library style” cataloging to large electronic document collections, as well as those developing automated or mixed methods, to directly compare their proposals for indexing and retrieval. This article concludes by suggesting guidelines for constructing such a testbed.

Current acoustic speech recognition technology performs well with very small vocabularies in noise or with large vocabularies in very low noise. Accurate acoustic speech recognition in noise with vocabularies over 100 words has yet to be achieved. Humans frequently lipread the visible facial speech articulations to enhance speech recognition, especially when the acoustic signal is degraded by noise or hearing impairment. Automatic lipreading has been found to improve significantly acoustic speech recognition and could be advantageous in noisy environments such as offices, aircraft and factories.