In this work we describe the results of a large-scale study on the effect of the distribution of labels across the different grades of relevance in the training set on the performance of trained ranking functions. In a controlled experiment we generate a large number of training datasets wih different label distributions and employ three learning to rank algo- rithms over these datasets. We investigate the effect of these distributions on the accuracy of obtained ranking functions to give an insight into the manner training sets should be constructed.

Learning-to-rank has attracted great attention in the IR community. Much thought and research has been placed on query-document feature extraction and development of sophisticated learning-to-rank algorithms. However, relatively little research has been conducted on selecting documents for learning-to-rank data sets nor on the effect of these choices on the efficiency and effectiveness of learning-to-rank algorithms.

Many research implementations of search engines are written in C, C++, or Java. They are difficult to understand and modify because they are at least a few thousand lines of code and contain many low-level details. In this paper, we show how to achieve a much shorter and higher level implementation: one in about a few hundred lines. We accomplish this result through the use of a high-level functional programming language, F#, and some of its features such as sequences, pipes and structured input and output. By using a search engine implementation as a case study, we argue that functional programming fits the domain of Information Retrieval problems much better than imperative/OO languages like C++ and Java. Functional programming languages are ideal for rapid algorithm prototyping and data exploration in the field of Information Retrieval (IR). Additionally, our implementation can be used as case study in an IR course since it is a very high level, but nevertheless executable specification of a search engine.

This paper evaluates the uptake and efficacy of a unified approach to phrasal query suggestions in the context of a high-precision search engine. The search engine performs ranked extended-Boolean searches with the proximity operator NEAR being the default operation. Suggestions are offered to the searcher when the length of the result list falls outside predefined bounds. If the list is too long, the engine suggests narrowing the query through the use of super phrases; if the list is too short, the engine suggests broadening the query through the use of proximal subphrases.