RULER: Robust Unified LLM-based Efficient Retrieval for Legal Information
sigir26-174 · Full research paper · Chenyu Hou, Z B Wang, Bin Cao, Jiaxing Wang, Tianming Zhang, Tiantian Li
Domain-Specific and Structured Retri · 50% Cross-Encoder and Neural Reranking · 50%
Abstract
Legal information retrieval demands high precision, yet traditional ''Retrieve-then-Rerank'' pipelines with two separate models suffer from cascading error propagation and knowledge disconnects between stages. To address these issues, we propose RULER, a Robust Unified LLM-based Efficient Retrieval that integrates efficient Bi-Encoder retrieval and high-precision Cross-Encoder reranking within a parameter-sharing architecture. To mitigate the Phantom Hits problem that irrelevant documents are assigned unreasonably high confidence, we introduce a Distribution-Robust Data Construction strategy that explicitly simulates pure-negative candidate groups. This is coupled with a Dynamic Margin Ranking Objective and Maximum Entropy Regularization, which collectively enforce uncertainty on irrelevant samples and enhance robustness. Extensive experiments on the JuDGE and LeCaRDv2 benchmarks demonstrate that RULER achieves state-of-the-art performance, outperforming all independent retrievers in retrieval tasks and surpassing competing unified architectures—where retriever and reranker parameters are shared—in high-precision reranking.
Assigned reviewers
| # | Reviewer | Match | Why (closest evidence publications in the taxonomy) | |
|---|---|---|---|---|
| 1 | Yuchen Li 0006 | 43 | 47 ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability (2026) LLM-Based Reranking 46 S2phere: Semi-Supervised Pre-training for Web Search over Heterogeneous Learning to Rank Data (2023) Web-Scale Learning to Rank 45 MPGraf: a Modular and Pre-trained Graphformer for Learning to Rank at Web-scale (2023) Web-Scale Learning to Rank |
|
| 2 | Tingyu Song | 40 | 51 IFIR: A Comprehensive Benchmark for Evaluating Instruction-Following in Expert-Domain Information Retrieval (2025) User Behavior and Search Interaction 43 LimRank: Less is More for Reasoning-Intensive Information Reranking (2025) Evidence Selection and Context Refinement 37 Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems (2026) Reasoning-Intensive Retrieval and Ranking |
Area chair
| Area chair | Match | |
|---|---|---|
| Chenyan Xiong | 46 |
Area-chair candidates (7)
| Chenyan Xiong assigned | 46 | 15 papers | |
| Zhijing Wu 0001 | 43 | 9 papers | |
| Xiaoyi Zeng | 39 | 11 papers | |
| Fernando Diaz 0001 | 38 | 12 papers | |
| Hongjin Qian | 38 | 12 papers | |
| Xiaoxue Zang | 38 | 4 papers | |
| Jun Xu 0001 | 37 | 2 papers |
Candidates from the panel ranked by match score
| # | Reviewer | Match | Load | Why | |
|---|---|---|---|---|---|
| 1 | Jurek Leonhardt | 50 | 3 | 64 Extractive Explanations for Interpretable Text Ranking (2023) Cross-Encoder and Neural Reranking 44 Data Augmentation for Sample Efficient and Robust Document Ranking (2024) Ranking Distillation and Adaptation |
|
| 2 | Shengyao Zhuang | 49 | 3 | 72 TILDE: Term Independent Likelihood moDEl for Passage Re-ranking (2021) Efficient Retrieval and Indexing 57 Dealing with Typos for BERT-based Passage Retrieval and Ranking (2021) Robustness and Adversarial IR |
|
| 4 | Jingtao Zhan | 44 | 3 | 67 An Analysis of BERT in Document Ranking (2020) Sparse and Dense Retrieval Encoders 64 Leveraging Passage-level Cumulative Gain for Document Ranking (2020) Cross-Encoder and Neural Reranking |
|
| 5 | Keping Bi | 43 | 3 | 58 Pre-training with Aspect-Content Text Mutual Prediction for Multi-Aspect Dense Retrieval (2023) Sparse and Dense Retrieval Encoders 57 A Multi-Granularity-Aware Aspect Learning Model for Multi-Aspect Dense Retrieval (2024) Sparse and Dense Retrieval Encoders |
|
| 6 | Yuchen Li 0006 assigned | 43 | 3 | 47 ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability (2026) LLM-Based Reranking 46 S2phere: Semi-Supervised Pre-training for Web Search over Heterogeneous Learning to Rank Data (2023) Web-Scale Learning to Rank |
|
| 8 | Xinyu Ma 0001 | 42 | 3 | 57 Pre-training with Aspect-Content Text Mutual Prediction for Multi-Aspect Dense Retrieval (2023) Sparse and Dense Retrieval Encoders 49 Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents (2023) Reasoning-Intensive Retrieval and Ranking |
|
| 9 | Xueguang Ma | 42 | 3 | 49 Fine-Tuning LLaMA for Multi-Stage Text Retrieval (2024) Sparse and Dense Retrieval Encoders 47 R2LLMs: Retrieval and Ranking with LLMs (2025) Reasoning-Intensive Retrieval and Ranking |
|
| 10 | Venktesh V | 42 | 3 | 66 Reproducing Adaptive Reranking for Reasoning-Intensive IR (2026) Evidence Selection and Context Refinement 54 Breaking the Lens of the Telescope: Online Relevance Estimation over Large Retrieval Sets (2025) Learned Sparse Retrieval Efficiency |
|
| 11 | Sheng-Chieh Lin | 41 | 3 | 55 One Blade for One Purpose: Advancing Math Information Retrieval using Hybrid Search (2023) Learned Sparse Retrieval Efficiency 54 Improving Conversational Passage Re-ranking with View Ensemble (2023) Conversational Search and Retrieval |
|
| 12 | Antonio Mallia | 41 | 3 | 54 E2Rank: Efficient and Effective Layer-Wise Reranking (2025) Learned Sparse Retrieval Efficiency 40 Fast Disjunctive Candidate Generation Using Live Block Filtering (2021) Efficient Retrieval and Indexing |
|
| 13 | Franco Maria Nardini | 41 | 3 | 54 Efficient Re-ranking with Cross-encoders via Early Exit (2025) Learned Sparse Retrieval Efficiency 54 Quality versus efficiency in document scoring with learning-to-rank models (2016) Learned Sparse Retrieval Efficiency |
|
| 14 | Zhen Qin 0001 | 41 | 3 | 75 Multi-Task Learning for Email Search Ranking with Auxiliary Query Clustering (2018) Domain-Specific and Structured Retrieval 72 Can Query Expansion Improve Generalization of Strong Cross-Encoder Rankers? (2024) Query Reformulation and Expansion |
|
| 15 | Xiao Wang 0043 | 41 | 3 | 41 Neural Pseudo-Relevance Feedback Models for Sparse and Dense Retrieval (2022) Query Reformulation and Expansion 41 ColBERT-PRF: Semantic Pseudo-Relevance Feedback for Dense Passage and Document Retrieval (2023) Query Reformulation and Expansion |
|
| 16 | Jing Liu 0022 | 40 | 3 | 64 A Hierarchical Entity-Based Approach to Structuralize User Generated Content in Social Media: A Case of Yahoo! Answers (2013) Domain-Specific and Structured Retrieval 47 Self-Calibrated Listwise Reranking with Large Language Models (2025) LLM-Based Reranking |
|
| 17 | Shuaiqiang Wang | 40 | 3 | 45 GS2P: a generative pre-trained learning to rank model with over-parameterization for web-scale search (2024) Web-Scale Learning to Rank 45 GS2P: A Generative Pre-trained Learning to Rank Model with Over-parameterization for Web-Scale Search (Extended Abstract) (2024) Web-Scale Learning to Rank |
|
| 18 | Tingyu Song assigned | 40 | 3 | 51 IFIR: A Comprehensive Benchmark for Evaluating Instruction-Following in Expert-Domain Information Retrieval (2025) User Behavior and Search Interaction 43 LimRank: Less is More for Reasoning-Intensive Information Reranking (2025) Evidence Selection and Context Refinement |
|
| 19 | Ruqing Zhang 0001 | 40 | 3 | 40 B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders 40 PROP: Pre-training with Representative Words Prediction for Ad-hoc Retrieval (2021) Sparse and Dense Retrieval Encoders |
|
| 20 | Rolf Jagerman | 40 | 3 | 72 Can Query Expansion Improve Generalization of Strong Cross-Encoder Rankers? (2024) Query Reformulation and Expansion 65 RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses (2023) Cross-Encoder and Neural Reranking |
|
| 21 | Leif Azzopardi | 39 | 3 | 64 Broad expertise retrieval in sparse data environments (2007) Domain-Specific and Structured Retrieval 64 A language modeling framework for expert finding (2009) Domain-Specific and Structured Retrieval |
|
| 22 | Corby Rosset | 39 | 3 | 54 Contextual Re-Ranking with Behavior Aware Transformers (2020) Conversational Search and Retrieval 40 Optimizing Query Evaluations Using Reinforcement Learning for Web Search (2018) Efficient Retrieval and Indexing |
|
| 23 | Andrew Parry | 39 | 3 | 70 Optimal Re-Ranking Depth (2026) Query Performance Prediction 43 "In-Context Learning" or: How I learned to stop worrying and love "Applied Information Retrieval" (2024) Sparse and Dense Retrieval Encoders |
|
| 24 | Sebastian Bruch 0001 | 39 | 3 | 45 Training Efficient Tree-Based Models for Document Ranking (2013) Web-Scale Learning to Rank 42 TF-Ranking: Scalable TensorFlow Library for Learning-to-Rank (2019) Retrieval Toolkits and Reproducibility |
|
| 25 | Adam Jatowt | 39 | 3 | 73 TempRetriever: Fusion-based Temporal Dense Passage Retrieval for Time-Sensitive Questions (2026) Domain-Specific and Structured Retrieval 52 BracketRank: Large Language Model Document Ranking via Reasoning-based Competitive Elimination (2026) Reasoning-Intensive Retrieval and Ranking |
|
| 27 | Weiwei Sun 0001 | 39 | 3 | 52 TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategy (2025) Reasoning-Intensive Retrieval and Ranking 49 Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents (2023) Reasoning-Intensive Retrieval and Ranking |
|
| 28 | Daiting Shi | 38 | 3 | 47 TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategy (2025) LLM-Based Reranking 43 Pre-trained Language Model based Ranking in Baidu Search (2021) Learned Sparse Retrieval Efficiency |
|
| 29 | Yanzhao Zhang | 38 | 3 | 69 SSRB: Direct Natural Language Querying to Massive Heterogeneous Semi-Structured Data (2025) Domain-Specific and Structured Retrieval 49 ERank: Fusing Supervised Fine-Tuning and Reinforcement Learning for Effective and Efficient Text Reranking (2026) LLM-Based Reranking |
|
| 31 | Xuanhui Wang COI: co-author | 38 | 3 | 72 Can Query Expansion Improve Generalization of Strong Cross-Encoder Rankers? (2024) Query Reformulation and Expansion 47 Learning to rank with multi-aspect relevance for vertical search (2012) Personalized and Context-Aware Retrieval |
|
| 32 | Chen Zhao 0013 | 38 | 3 | 47 LimRank: Less is More for Reasoning-Intensive Information Reranking (2025) LLM-Based Reranking 42 Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval (2021) Multi-hop and Complex Reasoning QA |
|
| 33 | Negar Arabzadeh | 38 | 3 | 41 Learning Query-Space Document Representations for High-Recall Retrieval (2023) Dense Bi-Encoder Training and Representation Learning 39 Enhanced Retrieval Effectiveness through Selective Query Generation (2024) Query Reformulation and Expansion |
|
| 36 | Yuzhong Qu | 37 | 3 | 64 Towards More Usable Dataset Search: From Query Characterization to Snippet Generation (2019) Domain-Specific and Structured Retrieval 33 Enhancing Complex Question Answering over Knowledge Graphs through Evidence Pattern Retrieval (2024) Knowledge Graph Question Answering |