Abstract In this study, we investigate information retrieval (IR) on Turkish texts using a large‐scale test collection that contains 408,305 documents and 72 ad hoc queries. We examine the effects of several stemming options and query‐document matching functions on retrieval performance. We show that a simple word truncation approach, a word truncation approach that uses language‐dependent corpus statistics, and an elaborate lemmatizer‐based stemmer provide similar retrieval effectiveness in Turkish IR. We investigate the effects of a range of search conditions on the retrieval performance; these include scalability issues, query and document length effects, and the use of stopword list in indexing.

We present the results of the first large-scale Turkish information retrieval experiments performed on a TREC-like test collection. The test bed, which has been created for this study, contains 95.5 million words, 408,305 documents, 72 ad hoc queries and has a size of about 800MB. All documents come from the Turkish newspaper Milliyet. We implement and apply simple to sophisticated stemmers and various query-document matching functions and show that truncating words at a prefix length of 5 creates an effective retrieval environment in Turkish. However, a lemmatizer-based stemmer provides significantly better effectiveness over a variety of matching functions.