Text ranking has witnessed significant advancements, attributed to the utilization of dualencoder enhanced by Pre-trained Language Models (PLMs).Given the proliferation of available PLMs, selecting the most effective one for a given dataset has become a non-trivial challenge.As a promising alternative to human intuition and brute-force fine-tuning, Transferability Estimation (TE) has emerged as an effective approach to model selection.However, current TE methods are primarily designed for classification tasks, and their estimated transferability may not align well with the objectives of text ranking.To address this challenge, we propose to compute the expected rank as transferability, explicitly reflecting the model's ranking capability.Furthermore, to mitigate anisotropy and incorporate training dynamics, we adaptively scale isotropic sentence embeddings to yield an accurate expected rank score.Our resulting method, Adaptive Ranking Transferability (AiRTran), can effectively capture subtle differences between models.On challenging model selection scenarios across various text ranking datasets, it demonstrates significant improvements over previous classificationoriented TE methods, human intuition, and ChatGPT with minor time consumption.

Text retrieval, a pivotal application in Natural Language Processing (NLP), involves retrieving pertinent documents from a candidate pool for a certain query. Due to the outstanding performance and improved convergence, Pre-trained Language Models (PLMs) are extensively employed in text retrieval. However, selecting the prime model from a multitude of PLMs remains a challenge, termed Model Selection (MS). In this paper, we address the MS by proposing a novel two-stage approach, Teaching and Exploiting Large Language Models (TELLMe). In the first stage, we efficiently retrieve models through Large Language Models (LLMs) prompting, leveraging the unparalleled world knowledge of LLM, where In-Context Learning (ICL) and Chain-of-Thought (CoT) are incorporated to overcome the inherent challenges in LLMs’ understanding of MS tasks. After retrieving potential high-performing models, the second stage employs Transferability Estimation (TE) for further model ranking. To tackle the issue of untrue hard labels faced by TE approaches, we propose a novel method, Evidence maximized from Soft labels (EaSe), which utilizes soft labels from LLM-encoded query embeddings and document embeddings to evaluate transferability scores. Our proposed approach combines the exceptional capabilities of LLMs and provides an efficient and generalized solution for MS regarding text retrieval. We further systematically evaluate the TELLMe framework on three diverse datasets, demonstrating its effectiveness and superiority.