Prompt compression for Retrieval-Augmented Generation (RAG) often fails by treating all retrieved information uniformly. This undifferentiated approach neglects the critical distinction between foundational core knowledge and illustrative practical knowledge, a failure especially damaging in hierarchical domains like law where essential principles can be discarded for redundant details, diminishing information gain.

In legal case retrieval, existing work has shown that human-mediated conversational search can improve users’ search experience. One of the key problems for a practical conversational search system is how to ask high-quality clarifying questions to initiate conversations with users and understand their search intents. Previous works demonstrated that human-annotated external domain knowledge (such as event schemas) can improve the legal utility of clarifying questions generated by large language models. However, these methods are restricted to specific law systems or languages and cannot be generalized to others. To this end, we propose to generate context and domain-specific questions with LLMs without external annotations or knowledge by extracting information from top-retrieved documents given the current conversation context. Specifically, we construct a conversational legal case retrieval system CARQ that iteratively selects neighbor candidate case documents from the retrieved list at each conversation step to ask clarifying questions. We pretrain CARQ to capture the differences between legal cases and employ the reward augmented maximum likelihood to optimize the system directly for retrieval metrics. Extensive automated and human evaluations on three widely adopted legal case retrieval datasets demonstrate the superior effectiveness of our approach as compared with the state-of-the-art baselines.

Legal case retrieval is an important task in information retrieval that aims to retrieve relevant cases for given query cases. Conversational search paradigms have been shown to improve the search experience in legal case retrieval. However, there are two challenges in applying conversational search to legal scenarios. Firstly, legal search conversations often focus on different parts of legal case documents, but existing models struggle to capture the complex structural information and extract accurate relevance signals. Secondly, collecting large-scale conversational search datasets is costly, making it difficult to build reliable conversational legal case retrieval models. To address these challenges, we propose a Structure-Aware Matching Model (SAMM) for conversational legal case retrieval. SAMM extracts matching signals between conversational utterances and segments of the legal cases to incorporate structural information. We decouple the conversational search task into three subtasks and design pre-training tasks to overcome the lack of training data. Additionally, we create ConvLegal, the largest conversational legal case retrieval dataset to the best of our knowledge, for better evaluation of different methods. We train and evaluate SAMM and baselines on both a public dataset (CLCR) and ConvLegal. Experimental results demonstrate that SAMM outperforms existing models in legal case retrieval and conversational search.

Similar case retrieval (SCR) is a representative legal AI application that plays a pivotal role in promoting judicial fairness. However, existing SCR datasets only focus on the fact description section when judging the similarity between cases, ignoring other valuable sections (e.g., the court's opinion) that can provide insightful reasoning process behind. Furthermore, the case similarities are typically measured solely by the textual semantics of the fact descriptions, which may fail to capture the full complexity of legal cases from the perspective of legal knowledge. In this work, we present MUSER, a similar case retrieval dataset based on multi-view similarity measurement and comprehensive legal element with sentence-level legal element annotations. Specifically, we select three perspectives (legal fact, dispute focus, and law statutory) and build a comprehensive and structured label schema of legal elements for each of them, to enable accurate and knowledgeable evaluation of case similarities. The constructed dataset originates from Chinese civil cases and contains 100 query cases and 4,024 candidate cases. We implement several text classification algorithms for legal element prediction and various retrieval methods for retrieving similar cases on MUSER. The experimental results indicate that incorporating legal elements can benefit the performance of SCR models, but further efforts are still required to address the remaining challenges posed by MUSER. The source code and dataset are released at https://github.com/THUlawtech/MUSER.

Legal case retrieval is a special IR task aiming to retrieve supporting cases for a given query case. Existing works have shown that conversational search paradigm can improve users' search experience in legal case retrieval. One of the keys to a practical conversational search system is how to ask high-quality clarifying questions to initiate conversations with users and understand their search intents. Recently, Large Language Models, such as ChatGPT and GPT-4, have shown superior ability in both open-domain QA and conversations with human. Thus it is natural to believe that they could be applied to legal conversational search as well. However, our preliminary study has shown that generating clarifying questions in legal conversational search with SOTA LLMs (e.g., GPT-4) often suffers from several problems such as duplication and low-utility contents. To address these problems, we propose LeClari, which leverages legal event schema as external knowledge to instruct LLMs to generate effective clarifying questions for legal conversational search. LeClari is constructed with a prompt module and a novel legal event selection module. The former defines a prompt with legal events for clarifying question generation and the latter selects potential event types by modeling the relationships of legal event types, conversational context, and candidate cases. We also propose ranking-oriented rewards and employ the reward augmented maximum likelihood (RAML) method to optimize LeClari directly based on the final retrieval performance of the conversational legal search system. Empirical results over two widely adopted legal case retrieval datasets demonstrate the effectiveness of our approach as compared with the state-of-the-art baselines.