Information Retrieval has a long history of applying either discriminative or generative modeling to retrieval and ranking tasks. Recent developments in transformer architectures and multi-task learning techniques have dramatically improved our ability to train effective neural models capable of resolving a wide variety of tasks using either of these paradigms. In this paper, we propose a novel multi-task learning approach which can be used to produce more effective neural ranking models. The key idea is to improve the quality of the underlying transformer model by cross-training a retrieval task and one or more complementary language generation tasks. By targeting the training on the encoding layer in the transformer architecture, our experimental results show that the proposed multi-task learning approach consistently improves retrieval effectiveness on the targeted collection and can easily be re-targeted to new ranking tasks. We provide an in-depth analysis showing how multi-task learning modifies model behaviors, resulting in more general models.

Thinking in terms of an information need instead of simply queries provides a rich set of new opportunities in improving the effectiveness of search [6]. User queries may vary a lot for a single information need [3, 9], as a query is often under-specified. Many techniques have been proposed to enrich a single query, for example relevance modeling [8]. These techniques focus on improving overall system performance but may fail in some occasions. Instead of optimizing for a single query, another direction is to use multiple query variations to represent an information need. With fusion techniques, query variations can improve system performance while failing fewer queries [2, 6].