Abstract Evaluating natural language generation systems is challenging due to the diversity of valid outputs. While human evaluation is the gold standard, it suffers from inconsistencies, lack of standardization, and demographic biases, limiting reproducibility. LLM-based evaluators offer a scalable alternative but are highly sensitive to prompt design, where small variations can lead to significant discrepancies. In this work, we propose an inversion learning method that learns effective reverse mappings from model outputs back to their input instructions, enabling the automatic generation of highly effective, model-specific evaluation prompts. Our method requires only a single evaluation sample and eliminates the need for time-consuming manual prompt engineering, thereby improving both efficiency and robustness. Our work contributes toward a new direction for more robust and efficient LLM-based evaluation.

Text ranking has witnessed significant advancements, attributed to the utilization of dualencoder enhanced by Pre-trained Language Models (PLMs).Given the proliferation of available PLMs, selecting the most effective one for a given dataset has become a non-trivial challenge.As a promising alternative to human intuition and brute-force fine-tuning, Transferability Estimation (TE) has emerged as an effective approach to model selection.However, current TE methods are primarily designed for classification tasks, and their estimated transferability may not align well with the objectives of text ranking.To address this challenge, we propose to compute the expected rank as transferability, explicitly reflecting the model's ranking capability.Furthermore, to mitigate anisotropy and incorporate training dynamics, we adaptively scale isotropic sentence embeddings to yield an accurate expected rank score.Our resulting method, Adaptive Ranking Transferability (AiRTran), can effectively capture subtle differences between models.On challenging model selection scenarios across various text ranking datasets, it demonstrates significant improvements over previous classificationoriented TE methods, human intuition, and ChatGPT with minor time consumption.

Identifying sentences sharing similar meanings is crucial to speech and text understandings. Although currently popular cross-encoder solutions with pre-trained language models as backbone have achieved remarkable performance, they suffer from the lack of the permutation invariance or symmetry that is one of the most important inductive biases to such task. To alleviate this issue, in this research we propose a permutation invariant training framework, in which a symmetry regularization is introduced during training that forces the model to produce the same predictions for input sentence pairs in both forward and backward directions. Empirical studies exhibit improved performance over competitive baselines.