Computational models of legal reasoning often employ factors to reason about cases. Factors can be used to analogize a current factual scenario and precedents and to make arguments for or against a conclusion. Courts not only determine whether a factor applies to a case or not, but often how strongly the factor applies, that is, the factor’s magnitude in the case. Previous methods for automatically extracting factors from cases cannot identify factors’ magnitudes. We present and evaluate a method employing Large Language Models (LLMs) to identify factor magnitudes using few-shot prompts with or without Wordnet definitions. We also show how the extracted magnitudes can be used in constructing legal arguments that employ factors and magnitudes the way judges and lawyers do.

Two major challenges for legal argument mining are limited annotated data and the need to model context-dependent reasoning. This paper presents a framework that enhances few-shot in-context learning (ICL) for legal argument mining by combining context-augmented similarity with hard label diversity filtering. Our method selects example sentences not only based on their semantic closeness to the target but also by incorporating their surrounding context and ensuring diverse label representation. Our legal argument mining labels are Issue, Reason, Conclusion, and Non-IRC classes. We evaluate the interaction between different context configurations (pre-, post-, and bidirectional) and label diversity under various few-shot settings on a legal case summary annotation task. Results show that our combined approach outperforms traditional retrieval strategies across multiple large language models with improved macro F1 scores. This work demonstrates a way of improving legal argument mining in low-resource settings by optimizing the selection of few-shot examples.

Large language models (LLMs) with extended context windows show promise for complex legal reasoning tasks, yet their ability to understand long legal documents remains insufficiently evaluated. Developing long-context benchmarks that capture realistic, high-stakes tasks remains a significant challenge in the field, as most existing evaluations rely on simplified synthetic tasks that fail to represent the complexity of real-world document understanding. Overruling relationships are foundational to common-law doctrine and commonly found in judicial opinions. They provide a focused and important testbed for long-document legal understanding that closely resembles what legal professionals actually do. We present an assessment of state-of-the-art LLMs on identifying overruling relationships from U.S. Supreme Court cases using a dataset of 236 case pairs. Our evaluation reveals three critical limitations: (1) era sensitivity – the models show degraded performance on historical cases compared to modern ones, revealing fundamental temporal bias in their training; (2) shallow reasoning – models rely on shallow logical heuristics rather than deep legal comprehension; and (3) context-dependent reasoning failures – models produce temporally impossible relationships in complex open-ended tasks despite maintaining basic temporal awareness in simple contexts. Our work contributes a benchmark that addresses the critical gap in realistic long-context evaluation, providing an environment that mirrors the complexity and stakes of actual legal reasoning tasks. The full dataset can be accessed at https://github.com/lizhang-AIandLaw/Do-LLMs-Truly-Understand-When-a-Precedent-Is-Overruled

Factors are a foundational component of legal analysis and computational models of legal reasoning. These factor-based representations enable lawyers, judges, and AI and Law researchers to reason about legal cases. In this paper, we introduce a methodology that leverages large language models (LLMs) to discover lists of factors that effectively represent a legal domain. Our method takes as input raw court opinions and produces a set of factors and associated definitions. We demonstrate that a semi-automated approach, incorporating minimal human involvement, produces factor representations that can predict case outcomes with moderate success, if not yet as well as expert-defined factors can.

This research addresses how to automatically identify certain factors in the texts of legal decisions and analyze their role in courts' decisions. It focuses on drug interdiction auto stop cases in which courts decide whether police officers have reasonable suspicion to detain a motorist. It illustrates how the methods to identify factors automatically can support empirical legal research in the domain and how machine learning methods of different accuracy and interpretability can be harnessed to explain case outcomes in terms legal professionals can understand.

Manual annotation is just as burdensome as it is necessary for some legal text analytic tasks. Given the promising performance of Generative Pretrained Transformers (GPT) on a number of different tasks in the legal domain, it is natural to ask if it can help with text annotation. Here we report a series of experiments using GPT-4 and GPT 3.5 as a pre-annotation tool to determine whether a sentence in a legal opinion describes a legal factor. These GPT models assign labels that human annotators subsequently confirm or reject. To assess the utility of pre-annotating sentences at scale, we examine the agreement among gold-standard annotations, GPT's pre-annotations, and law students' annotations. The agreements among these groups support that using GPT-4 as a pre-annotation tool is a useful starting point for large-scale annotation of factors.

Traditional evaluation metrics like ROUGE compare lexical overlap between the reference and generated summaries without taking argumentative structure into account, which is important for legal summaries. In this paper, we propose a novel legal summarization evaluation framework that utilizes GPT-4 to generate a set of question-answer pairs that cover main points and information in the reference summary. GPT-4 is then used to produce answers based on the generated summary for the questions from the reference summary. Finally, GPT-4 grades the answers from the reference summary and the generated summary. We examined the correlation between GPT-4 grading and human grading. The results suggest that this question-answering approach with GPT-4 can be a useful tool for gauging the quality of the summary.

In making legal decisions, courts apply relevant law to facts. While the law typically changes slowly over time, facts vary from case to case. Nevertheless, underlying patterns of fact may emerge. This research focuses on underlying fact patterns commonly present in cases where motorists are stopped for a traffic violation and subsequently detained while a police officer conducts a canine sniff of the vehicle for drugs. We present a set of underlying patterns of fact, that is, factors of suspicion, that police and courts apply in determining reasonable suspicion. We demonstrate how these fact patterns can be identified and annotated in legal cases and how these annotations can be employed to fine-tune a transformer model to identify the factors in previously unseen legal opinions.

We propose an adaptive environment (CABINET) to support caselaw analysis (identifying key argument elements) based on a novel cognitive computing framework that carefully matches various machine learning (ML) capabilities to the proficiency of a user. CABINET supports law students in their learning as well as professionals in their work. The results of our experiments focused on the feasibility of the proposed framework are promising. We show that the system is capable of identifying a potential error in the analysis with very low false positives rate (2.0–3.5%), as well as of predicting the key argument element type (e.g., an issue or a holding) with a reasonably high F1-score (0.74).

In this paper, we explore legal argument mining using multiple levels of granularity. Argument mining has usually been conceptualized as a sentence classification problem. In this work, we conceptualize argument mining as a token-level (i.e., word-level) classification problem. We use a Longformer model to classify the tokens. Results show that token-level text classification identifies certain legal argument elements more accurately than sentence-level text classification. Token-level classification also provides greater flexibility to analyze legal texts and to gain more insight into what the model focuses on when processing a large amount of input data.

In this paper, we examine the use of multi-lingual sentence embeddings to transfer predictive models for functional segmentation of adjudicatory decisions across jurisdictions, legal systems (common and civil law), languages, and domains (i.e. contexts). Mechanisms for utilizing linguistic resources outside of their original context have significant potential benefits in AI & Law because differences between legal systems, languages, or traditions often block wider adoption of research outcomes. We analyze the use of Language-Agnostic Sentence Representations in sequence labeling models using Gated Recurrent Units (GRUs) that are transferable across languages. To investigate transfer between different contexts we developed an annotation scheme for functional segmentation of adjudicatory decisions. We found that models generalize beyond the contexts on which they were trained (e.g., a model trained on administrative decisions from the US can be applied to criminal law decisions from Italy). Further, we found that training the models on multiple contexts increases robustness and improves overall performance when evaluating on previously unseen contexts. Finally, we found that pooling the training data from all the contexts enhances the models' in-context performance.

In this paper, we assess the use of several deep learning classification algorithms as a step toward automatically preparing succinct summaries of legal decisions. Short case summaries that tease out the decision's argument structure by making explicit its issues, conclusions, and reasons (i.e., argument triples) could make it easier for the lay public and legal professionals to gain an insight into what the case is about. We have obtained a sizeable dataset of expert-crafted case summaries paired with full texts of the decisions issued by various Canadian courts. As the manual annotation of the full texts is prohibitively expensive, we explore various ways of leveraging the existing longer summaries which are much less time-consuming to annotate. We compare the performance of the systems trained on the annotations that are manually ported to the full texts from the summaries to the performance of the same systems trained on annotations that are projected from the summaries automatically. The results show the possibility of pursuing the automatic annotation in the future.

Machine learning research typically starts with a fixed data set created early in the process. The focus of the experiments is finding a model and training procedure that result in the best possible performance in terms of some selected evaluation metric. This paper explores how changes in a data set influence the measured performance of a model. Using three publicly available data sets from the legal domain, we investigate how changes to their size, the train/test splits, and the human labelling accuracy impact the performance of a trained deep learning classifier. Our experiments suggest that analyzing how data set properties affect performance can be an important step in improving the results of trained classifiers, and leads to better understanding of the obtained results.

In this paper, we treat sentence annotation as a classification task. We employ sequence-to-sequence models to take sentence position information into account in identifying case law sentences as issues, conclusions, or reasons. We also compare the legal domain specific sentence embedding with other general purpose sentence embeddings to gauge the effect of legal domain knowledge, captured during pre-training, on text classification. We deployed the models on both summaries and full-text decisions. We found that the sentence position information is especially useful for full-text sentence classification. We also verified that legal domain specific sentence embeddings perform better, and that meta-sentence embedding can further enhance performance when sentence position information is included.

Human-performed annotation of sentences in legal documents is an important prerequisite to many machine learning based systems supporting legal tasks. Typically, the annotation is done sequentially, sentence by sentence, which is often time consuming and, hence, expensive. In this paper, we introduce a proof-of-concept system for annotating sentences “laterally.” The approach is based on the observation that sentences that are similar in meaning often have the same label in terms of a particular type system. We use this observation in allowing annotators to quickly view and annotate sentences that are semantically similar to a given sentence, across an entire corpus of documents. Here, we present the interface of the system and empirically evaluate the approach. The experiments show that lateral annotation has the potential to make the annotation process quicker and more consistent.

Argument mining, a subfield of natural language processing and text mining, is a process of extracting argumentative text portions and identifying the role the selected texts play. Legal argument mining targets the argumentative parts of a legal text. In order to better understand how to apply legal argument mining as a step toward improving case summarization, we have assembled a sizeable set of cases and human-expert-prepared summaries annotated in terms of legal argument triples that capture the most important skeletal argument structures in a case. We report the results of applying multiple machine learning techniques to demonstrate and analyze the advantages and disadvantages of different methods to identify sentence components of these legal argument triples.

Statutory texts employ vague terms that are difficult to understand. Here we study and evaluate methods for retrieving useful sentences from court opinions that elaborate on the meaning of a vague statutory term. Retrieving sentences instead of whole cases may spare a user the need to review long lists of cases in search of useful explanations. We assembled a data set of 4,635 sentences that were responses to three statutory queries and labeled them in terms of their usefulness for interpretation. We have run a series of experiments on this data set, which we have made public, assessing different techniques to solve the task. These include techniques that measure the similarity between the sentence and the query, utilize the context of a sentence, expand queries, or assess the novelty of a sentence with respect to a statutory provision from which the interpreted term comes. Based on a detailed error analysis we propose a specialized sentence retrieval framework that mitigates the challenges of retrieving case law sentences for interpreting statutory terms. The results of evaluating different implementations of the framework are promising (.725 for NDGC at 10, .662 at 100).

This paper reports results from the JusticeBot Project, in which we analyzed two datasets drawn from 1 million written decisions from the Régie du logement du Québec. Using an empirical methodology, we identified 44 factors that occur in disputes where the tenant seeks a remedy due to problems with the rented apartment, such as the existence of bedbugs, high noise levels or problems with insulation. In the first dataset, we used these factors to tag 149 cases. We found a correlation between how many factors are found in a case and how likely the judge is to award rent reduction to a tenant; the amount of reduction was also higher in cases with more factors. For the second dataset (39 cases with bedbugs, drawn from the first dataset), we developed in-depth factors and used them to tag the cases. We found a number of plausible correlations, such as the average damage award being higher in cases with infestations of high intensity. Finally, in predicting the decision of the judge using the factors present in a case, the results were similar to the baselines or slightly above. We discuss the possible reasons for this, and why the approach shows promise in providing useful information to lay people and lawyers.

We report on a pilot experiment in automatic, extractive summarization of legal cases concerning Post-traumatic Stress Disorder from the US Board of Veterans' Appeals. We hypothesize that length-constrained extractive summaries benefit from choosing among sentences that are predictive for the case outcome. We develop a novel train-attribute-mask pipeline using a CNN classifier to iteratively select predictive sentences from the case, which measurably improves prediction accuracy on partially masked decisions. We then select a subset for the summary through type classification, maximum marginal relevance, and a summarization template. We use ROUGE metrics and a qualitative survey to evaluate generated summaries along with expert-extracted and expert-drafted summaries. We show that sentence predictiveness does not reliably cover all decision-relevant aspects of a case, illustrate that lexical overlap metrics are not well suited for evaluating legal summaries, and suggest that future work should focus on case-aspect coverage.

In common law jurisdictions, legal research often involves an analysis of relevant case law. Court opinions comprise several high-level parts with different functions. A statement's membership in one of the parts is a key factor influencing how the statement should be understood. In this paper we present a number of experiments in automatically segmenting court opinions into the functional and the issue specific parts. We defined a set of seven types including Background, Analysis, and Conclusions. We used the types to annotate a sizable corpus of US trade secret and cyber crime decisions. We used the data set to investigate the feasibility of recognizing the parts automatically. The proposed framework based on conditional random fields proved to be very promising in this respect. To support research in automatic case law analysis we plan to release the data set to the public.

Vector Space Models (VSMs) represent documents as points in a vector space derived from term frequencies in the corpus. This level of abstraction provides a flexible way to represent complex semantic concepts through vectors, matrices, and higher-order tensors. In this paper we utilize a number of VSMs on a corpus of judicial decisions in order to classify cases in terms of legal factors, stereotypical fact patterns that tend to strengthen or weaken a side's argument in a legal claim. We apply different VSMs to a corpus of trade secret misappropriation cases and compare their classification results. The experiment shows that simple binary VSMs work better than previously reported techniques but that more complex VSMs including dimensionality reduction techniques do not improve performance.

Case law analysis is a significant component of research on almost any legal issue and understanding which agents are involved and mentioned in a decision is integral part of the analysis. In this paper we present a first experiment in detecting mentions of different agents in court decisions automatically. We defined a light-weight and easily extensible hierarchy of agents that play important roles in the decisions. We used the types from the hierarchy to annotate a corpus of US court decisions. The resulting data set enabled us to test the hypothesis that the mentions of agents in the decisions could be detected automatically. Conditional random fields models trained on the data set were shown to be very promising in this respect. To support research in automatic case-law analysis we release the agent mentions data set with this paper.

We report on prototype experiments expanding on prior work [2] in retrieving and ranking vaccine injury decisions using semantic information and classifying sentences as legal rules or findings about vaccine-injury causation. Our positive results include that query element coverage features and aggregate citation information using a BM25-like score can improve ranking results, and that larger amounts of annotated sentence data improve classification performance. Negative observations include that LUIMA-specific sentence features do not impact sentence classification, and that synthetic oversampling improves classification only for the sparser of the two predicted sentence types.

This paper presents first results from a proof of feasibility experiment in conceptual legal document retrieval in a particular domain (involving vaccine injury compensation). The conceptual markup of documents is done automatically using LUIMA, a law-specific semantic extraction toolbox based on the UIMA framework. The system consists of modules for automatic sub-sentence level annotation, machine learning based sentence annotation, basic retrieval using Apache Lucene and a machine learning based reranking of retrieved documents. In a leave-one-out experiment on a limited corpus, the resulting rankings scored higher for most tested queries than baseline rankings created using a commercial full-text legal information system.

In this paper we use statistical machine learning to classify statutory texts in terms of highly specific functional categories. We focus on regulatory provisions from multiple US state jurisdictions, all dealing with the same general topic of public health system emergency preparedness and response. In prior work we have established that one can improve classification performance on one jurisdiction's statutory texts using texts from another jurisdiction. Here we describe a framework facilitating transfer of predictive models for classification of statutory texts among multiple state jurisdictions. Our results show that the classification performance improves as we employ an increasing number of models trained on data coming from different states.

Statutory analysis is a significant component of research on almost any legal issue and determining if a statutory provision applies is an integral part of the analysis. In this paper we present the initial results from an attempt to support the applicability assessment in situations where the number of statutory provisions to be considered is large. We propose the use of a framework in which a single human expert cooperates with a machine learning text classification algorithm. Our experiments show that an adoption of the approach leads to a better performance during the relevance assessment. In addition, we suggest how to re-use a classification model trained during one statutory analysis for another related analysis. This points to a new way of capturing and re-using knowledge produced in the course of statutory analysis. Our experiments confirm the viability of this approach.

In this paper we mine statutory texts for highly-specific functional information using NLP techniques and a supervised ML approach. We focus on regulatory provisions from multiple state jurisdictions (Pennsylvania and Florida), all dealing with the same general topic (i.e., public health system emergency preparedness and response). While the number of annotated provisions from any one jurisdiction is not large, we are investigating whether one can improve classification performance on one jurisdiction's statutory texts by including other jurisdictions' annotated statutory texts dealing with the same general topic. Our experiments suggest that data from one jurisdiction can be used to boost the performance of the classifiers trained for different jurisdictions.

This paper explores how to extract argumentation-relevant information automatically from a corpus of legal decision documents, and how to build new arguments using that information. For decision texts, we use the Vaccine/Injury Project (V/IP) Corpus, which contains default-logic annotations of argument structure. We supplement this with presuppositional annotations about entities, events, and relations that play important roles in argumentation, and about the level of confidence that arguments would be successful. We then propose how to integrate these semantic-pragmatic annotations with syntactic and domain-general semantic annotations, such as those generated in the DeepQA architecture, and outline how to apply machine learning and scoring techniques similar to those used in the IBM Watson system for playing the Jeopardy! question-answer game. We replace this game-playing goal, however, with the goal of learning to construct legal arguments.

This paper expands on the previously published value judgment formalism. The representation of situations is enhanced by introducing event progressions similar to actions in general AI planning. Using event progressions, situations can be assessed as to what facts they contain as well as what facts may ensue with some likelihood, thereby opening up a situation space. Purposive legal argumentation can be modeled using propositions and rules controlling the likelihoods of value-laden consequences. The paper expands the formalism to cover event progressions and illustrates the functionality using an example based on Young v. Hitchens.

Users of commercial legal information retrieval (IR) systems often want argument retrieval (AR): retrieving not merely sentences with highlighted terms, but arguments and argument-related information. Using a corpus of argument-annotated legal cases, we conducted a baseline study of current legal IR systems in responding to standard queries. We identify ways in which they cannot meet the need for AR and illustrate how additional argument-relevant information could address some of those inadequacies. We conclude by indicating our approach to developing an AR system to retrieve arguments from legal decisions.

Argumentation is central to law. Written and oral argument structures, however, are often difficult to analyze and employ in instruction. Diagrammatic models of argument offer a potential solution to these problems. In this paper we report on the results of an empirical study into the diagnostic utility of argument diagrams in a legal writing context. The focus is on comparing experts' and student-produced argument diagrams and on the extent to which the latter can be used to predict students' performance on subsequent writing tasks. We present the results and draw some tentative conclusions.

This paper explains and illustrates in an example context how case comparison in legal case-based reasoning can be modeled in the value judgment formalism. It presents a set of argument schemes corresponding to typical moves in case-based reasoning which make use of intermediate legal concepts and their impact on the applicable values.

Given a renewed interest in the field of AI and Law in more complex factual representations of legal cases in terms of narratives, techniques for representing and reasoning about temporal orderings of facts will become increasingly important. The SIROCCO (System for Intelligent Retrieval of Operationalized Cases and COdes) program employed a representation for the temporal ordering of events in ethics cases in a way that informed determinations of whether and how ethical norms were violated and if the problem and other cases were normatively analogous at a deeper level. At the same time, the program supported ordinary case enterers in translating the facts of textually described cases into a machine-processable representation. This paper presents these previously unpublished aspects of the work including a report of an empirical evaluation of the contribution of the temporal representation to the program's success in retrieving relevant norms and cases. Although the results were negative, a consideration of the reasons why is illuminating. While SIROCCO dealt with engineering ethics cases, it is clear that similar temporal considerations apply in legal cases and that the approach is likely to be useful in legal narrative representations.

Applying Bayesian data analysis to model a computer-supported peer-review process in a legal class writing exercise yielded pedagogically useful information about student-understanding of problem-specific legal concepts and of more general domain-related legal writing criteria and about the criteria's effectiveness. The approach suggests how AI and Law can impact legal education.

This paper presents preliminary results in extracting semantic information from US state public health legislative provisions using natural language processing techniques and machine learning classifiers. Challenges in the density and distribution of the data as well as the structure of the prediction task are described. Decision tree models trained on a unigram representation with TFIDF measures in most cases outperform the baselines by varying margins, leaving room for further improvement.

This paper presents a formalism modeling legal reasoning with fact patterns and their substantive effects on legal values. It centers on the concept of a value judgment, i.e. a determination that one factual situation is preferable over another by virtue of their respective effects on values. This allows the modeling of legal sources as sets of value judgments and legal methodologies as collections of argumentation schemes. The paper briefly derives the formalism from legal theory and elaborates on its use in the context of an example of hypothetical reasoning.

This paper describes a study in which student-created diagrams about arguments in an ill-defined domain were manually graded by two independent human graders. Findings include that the graders overall agreed with each other on their grades, but their agreement was lower than one would expect in well-defined domains, and higher for solutions of extreme quality.

In 1993, Berman and Hafner criticized case-based models of legal reasoning for not modeling analogical and teleological elements. Another lesson learned since then is the role of ontologies in representing domain knowledge so that a legal reasoning system can represent and solve problems. If the reasoning involves drawing abstract analogies, reasoning teleologically about rules for deciding a case, and posing hypothetical cases to test decision rules, however, it is not clear what requirements the ontology should satisfy. This paper presents an extended example of such legal reasoning to illustrate what an ontology for case-based legal reasoning should provide. The example centers on a microworld of legal discourse, an ensemble of real legal cases, hypothetical examples, concepts, factors, principles and policies. Beginning with any case in the microworld, the system's goal is to generate arguments that a law professor and students might reasonably make in discussing the legal case in class. The example illustrates three roles the ontology should play in providing representational support for the system, distills the ontological requirements, and suggests an incremental approach to making good on Berman's and Hafner's challenge.

This paper outlines a process model for disambiguating legal provisions in order to ease their formalization into logic. It centers around a reformulation of the provision driven by critical questioning and mandatory legal justifications.

The development of graphical argument models is an active and growing area of research in Artificial Intelligence and Law. The aim is to develop models which may be readily used by legal professionals and novices to produce and parse arguments. If this goal is to be realized it is important to develop models that human reasoners can manipulate and assess consistently. We report on an ongoing study of graph agreement in the context of the LARGO system.

Diagrammatic models of argument are increasingly prominent in AI and Law. Unlike everyday language these models formalize many of the the components and relationships present in arguments and permit a more formal analysis of an arguments' structural weaknesses. Formalization, however, can raise problems of agreement. In order for argument diagramming to be widely accepted as a communications tool, individual authors and readers must be able to agree on the quality and meaning of a diagram as well as the role that key components play. This is especially problematic when arguers seek to map their diagrams to or from more conventional prose. In this paper we present results from a grader agreement study that we have conducted using LARGO diagrams. We then describe a detailed example of disagreement and highlight its implications for both our diagram model and modeling argument diagrams in general.

This paper presents a process model of arguing with hypotheticals and uses it to explain examples of oral arguments before the U.S. Supreme Court that are like those employed in Socratic law teaching. The process model has been partially implemented in the LARGO (Legal ARgument Graph Observer) intelligent tutoring system. The program supports students in diagramming oral argument examples; its feedback on students' diagrammatic reconstructions of the examples enforces the expectations of the process model. The paper presents empirical evidence that features of the argument diagrams made with LARGO are correlated with independent measures of argumentation ability. The examples and empirical results support the model's explanatory and diagnostic utility.

This paper describes an intelligent tutoring system, LARGO, that helps students learn skills of legal reasoning with hypotheticals by analyzing oral arguments before the US Supreme Court. The skills involve proposing a rule-like test for deciding a case, posing hypotheticals to challenge the rule, and responding by analogizing or distinguishing the hypotheticals and/or modifying the proposed test. Students diagram arguments in a special-purpose graphical language and receive feedback in the form of reflection questions.

The transcripts of oral arguments before the US Supreme Court provide interesting opportunities from the viewpoint of legal education. As the pinnacle of legal argumentation, they illustrate, often in dramatic fashion, a sophisticated process of concept formation and testing driven by skillful posing of hypotheticals. Yet it is not easy to get beginning law students to understand the arguments and the underlying processes of hypothesis formation and testing. We introduce a novel project with the dual aims of developing an AI model of concept formation and testing as well as an intelligent tutoring system for beginning law students. We describe a planned experiment in which we will evaluate to what extent law students' study of the Supreme Court oral arguments can be improved by providing detailed and specific self-explanation prompts. It is hypothesized that detailed prompts to explain connections between tests, rationales, dimensions, and hypotheticals will help students to induce adequate mental models of concept formation processes.

In this paper, we present methods for automatically finding abstract, legally relevant concepts in case texts and demonstrate how they can be used to make predictions of case outcomes, given the texts as inputs.In a set of experiments to test these methods, we focus on the open question of how best to represent legal text for finding abstract concepts. We compare different ways of representing legal case texts in order to test whether adding domain knowledge and some linguistic information can improve performance.We found that replacing individual names by roles in the case texts led to better indexing, and that adding certain syntactic and semantic information, in the form of Propositional Patterns that capture a sense of "who did what", led to better prediction. Our experiments also showed that of three learning algorithms, Nearest Neighbor worked best in learning how to identify indexing concepts in texts.In these experiments, we introduced a prototype system that can reason with text cases; it analyzes a case, predicts its outcome considering other cases in the database, and explains the prediction, all starting with a textual description of the case's facts as input.

In this paper, we introduce IBP, an algorithm that combines reasoning with an abstract domain model and case-based reasoning techniques to predict the outcome of case-based legal arguments. Unlike the predictions generated by statistical or machine-learning techniques, IBP's predictions are accompanied by explanations.We describe an empirical evaluation of IBP, in which we compare our algorithm to prediction based on Hypo's and CATO's relevance criteria, and to a number of widely used machine learning algorithms. IBP reaches higher accuracy than all competitors, and hypothesis testing shows that the observed differences are statistically significant. An ablation study indicates that both sources of knowledge in IBP contribute to the accuracy of its predictions.

This paper describes how we used an AI model for retrieving ethics cases to investigate empirically the epistemological contributions of a decision-makers' citing cases and code provisions in justifying decisions. In practical ethics, like law, it is impossible to define abstract principles intensionally so that they may be applied deductively. After investigating hundreds of professional ethics case opinions, we hypothesized that the decision-makers' explanations extensionally defined principles over time, in effect, operationalizing them. We constructed SIROCCO, a system for retrieving principles and past ethics cases. We used this computational model to conduct an ablation experiment concerning a core set of operationalization techniques. This paper presents empirical evidence that the operationalization information supports predictions of the relevant principles and past cases more accurately than competing approaches that do not use such information.

The prohibitive cost of assigning indices to textual cases is a major obstacle for the practical use of AI and Law systems supporting reasoning and arguing with cases. While progress has been made toward extracting certain facts from well-structured case texts or classifying case abstracts under Key Number concepts, these methods still do not suffice for the complexity of indexing concepts in CBR systems.

In this paper, we discuss the challenges in providing computer support for teaching professional ethics using a case-based approach. We describe our tutoring software, PETE, which helps students prepare cases for class discussion. PETE enables students to practice methods of moral reasoning. It also encourages them to compare their work to a range of other peer responses. We discuss how the program could incorporate AI techniques and how to evaluate its effectiveness.

CAT0 is an intelligent learning environment designed to help &@ting law students learn basic skills of making arguments with cases, through practice in theory-testing and argumentation tasks.CAT0 models ways in which experts compare and contrast cases, assess the significance of similarities and differences between cases in light of general domain knowledge, and use the same general knowledge to organize multi-case arguments by issues.CAT0 communicates its model to students by presenting dynamically-generated argumentation examples and reifying (i.e., making visible) argument structure.Also, the CAT0 Tools reduce some of the distracting complexity of the students' task.CAT0 employs a computational model of case-based argumentation that addresses eight basic argument moves and more elaborate multi-case arguments.The model includes a "Factor Hierarchy," which represents more nbstract.,but still domain-specific, legal knowledge about the meaning of the factors used to represent cases.CAT0 uses the Factor Hierarchy for a number of purposes, among them to organize multi-case arguments by issues and to make arguments about the significance of distinctions.To generate the latter, CAT0 strategically selects alternative interpretations of cases to elaborate a "deeper" (or more.abstract) contrast or parallel between cases.

This paper presents preliminary work towards automatically assigning to full-text opinion tezts the applicable factors, that is, fact patterns influencing the outcome of a legal claim, to full-text opinion tezts, which are used in CATO's model of case-based legal argumentation.In spite of the fundamentally difierent representation and methods for comparing cases in a CATO-style case-based reasoning system versus text-retrieval systems, the paper provides evidence that there exists a connection between the notion of similarity under both paradigms.We consider the task as a classification problem, and apply machine learning methods, where CATO's Case Database of 147 tmde secret law cases are used as training instances.Since the generalization power of purely inductive algorithms, which rely on representation from conventional text-retrieval, does not measure up to the wmplexity of the concepts corresponding to the factors, the learning algorithms'performance is not satisfactory yet.%ing to address this problem, we discuss techniques that will allow integrating domain specific knowledge, about the use of cases in legal argumentation.and about the interrelations among the factors, which is expressed in CAT05 Factor Hiemrchy.Our hypothesis is that adding this knowledge un'll enhance performance, and that a successful system m-11 facilitate building legal case-based reasoning systems and legal research.

We conducted an experiment to investigate whether a human tutor could employ the CATO model and instructional program to teach legal research and argumentation skills to beginning law stu&nts. The CATO model covem arguments comparing and contrasting casesin terms of factors, abstractionsof facts that tend to strengthen or weaken a party’s position on a legal claim. At the time of the experimen~ the CATO program comprised tools and resources that help apply the CATO model to specific problems, most importantly, a case database and tools for retrieving, displaying, and comparing casesin terms of factors. We compmed human-led itulruction with CATO against more traditional classroom instruction designed to teach the same skills, without the use of the CATO model or tools. The subjects were 17 fmtsemester students from the University of Pittsburgh Law School. We found that human-guided instruction with CATO was as good as classroom instruction, We also found that answers generated by the CATO program were scored higher than the students’ answers, suggestingthat the model can potentially be employed even more effectively to teach students. Examples drawn from protocols of CATO sessionsih.trate that students can use the CATO model to guide and facilitate the construction of arguments and often go beyond the model’s limitations, at least under the guidance of a human tutor.

Article Context sensitive case comparisons in practical ethics: reasoning about reasons Share on Authors: Bruce M. McLaren University of Pittsburgh, Intelligent Systems Program, Pittsburgh, Pennsylvania University of Pittsburgh, Intelligent Systems Program, Pittsburgh, PennsylvaniaView Profile , Kevin D. Ashley University of Pittsburgh, Intelligent Systems Program, Pittsburgh, Pennsylvania University of Pittsburgh, Intelligent Systems Program, Pittsburgh, PennsylvaniaView Profile Authors Info & Claims ICAIL '95: Proceedings of the 5th international conference on Artificial intelligence and lawMay 1995 Pages 316–325https://doi.org/10.1145/222092.222266Online:24 May 1995Publication History 5citation324DownloadsMetricsTotal Citations5Total Downloads324Last 12 Months3Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

To make legal arguments, one needs certain information about how to use cases effectively - dialectical information. In the broadest sense, dialectical information includes strategies for employing cases to justify legal conclusions (and responding to such justifications) and criteria for finding cases and deciding which cases to use. Making dialectical information explicit is important for teaching case-based argument. It is our experience that typically, law students do not have a very good set of dialectical strategies nor are they aware of the criteria. Even the most sophisticated legal information retrieval tools do not make such dialectical information explicit and assume that users have already learned it.

This paper describes a research project to devise and test an intelligent, case-baaed tutorial program for teaching law students to argue with cases.In order to present pedagogically interesting lessons and develop a Student Model, we have designed memory structures such as Argument Contexts and a hierarchy of Issues in Case-Baaed Legal Reaaoning.Using logical expressions in the knowledge representation language Loom, we also explicitly represent case-based argument concepts such as a case's being on point to a problem, more on point than another case, most on point of all the cases, a best case to cite, and a counterexample to another case.The program will be able to reason with the explicit concepts in selecting cases from a Case Library, assembling lessons and examples, analyzing student inputs, and in generating explanations and feedback.We hope to demonstrate empirically that, by providing law students a conceptual model of the criteria for selecting and describing precedents that would be useful in an argument, the tutorial program will help them to learn to select and apply cases more efficiently and to make more effective arguments.

This paper presents a partial theory of arguing with precedents in law and illustrates how that theory supports multiple interpretations of a precedent. The theory provides succinct computational definitions of (1) the most persuasive precedents to cite in the principal argument roles and (2) the most salient aspects of the precedents to emphasize when citing them in those roles. An extended example, drawn from the output of the HYPO program, illustrates the range of different descriptions of the same precedent that are supported by the theory. Each description focuses on different salient aspects of the case depending on the argument context.

An interesting and important aspect of legal reasoning is the use of citations to precedent cases as justifications for legal conclusions. In this paper, we describe the standard use of citations as described in the attorney's “Blue Book” and how HYPO, a program that models case-based legal reasoning, generates and uses citations in a very similar way to analyze fact situations and to communicate with an attorney/user. More specifically, we describe how, given a fact situation (“cfs”), HYPO dynamically generates the citations to cases in its Case Knowledge Base (“CKB”) by (1) analyzing the factual features of the cfs to see what dimensions apply, (2) retrieving and constructing a “neighborhood” of citable cases around the cfs (the “Claim Lattice”) and (3) constructing the “Cites Display”, a network of citations to the most on point cases (“mopc”) that is a skeletal frame for a legal argument about the cfs.

In this paper, we give an overview of our case-based reasoning program, HYPO, which operates in the field of trade secret law. We discuss key ingredients of case-based reasoning, in general, and the correspondence of these to elements of HYPO. We conclude with an extended example of HYPO working through a hypothetical trade secrets case, patterned after an actual case.