Computational models of legal reasoning often employ factors to reason about cases. Factors can be used to analogize a current factual scenario and precedents and to make arguments for or against a conclusion. Courts not only determine whether a factor applies to a case or not, but often how strongly the factor applies, that is, the factor’s magnitude in the case. Previous methods for automatically extracting factors from cases cannot identify factors’ magnitudes. We present and evaluate a method employing Large Language Models (LLMs) to identify factor magnitudes using few-shot prompts with or without Wordnet definitions. We also show how the extracted magnitudes can be used in constructing legal arguments that employ factors and magnitudes the way judges and lawyers do.

Factors are a foundational component of legal analysis and computational models of legal reasoning. These factor-based representations enable lawyers, judges, and AI and Law researchers to reason about legal cases. In this paper, we introduce a methodology that leverages large language models (LLMs) to discover lists of factors that effectively represent a legal domain. Our method takes as input raw court opinions and produces a set of factors and associated definitions. We demonstrate that a semi-automated approach, incorporating minimal human involvement, produces factor representations that can predict case outcomes with moderate success, if not yet as well as expert-defined factors can.

This research addresses how to automatically identify certain factors in the texts of legal decisions and analyze their role in courts' decisions. It focuses on drug interdiction auto stop cases in which courts decide whether police officers have reasonable suspicion to detain a motorist. It illustrates how the methods to identify factors automatically can support empirical legal research in the domain and how machine learning methods of different accuracy and interpretability can be harnessed to explain case outcomes in terms legal professionals can understand.

Manual annotation is just as burdensome as it is necessary for some legal text analytic tasks. Given the promising performance of Generative Pretrained Transformers (GPT) on a number of different tasks in the legal domain, it is natural to ask if it can help with text annotation. Here we report a series of experiments using GPT-4 and GPT 3.5 as a pre-annotation tool to determine whether a sentence in a legal opinion describes a legal factor. These GPT models assign labels that human annotators subsequently confirm or reject. To assess the utility of pre-annotating sentences at scale, we examine the agreement among gold-standard annotations, GPT's pre-annotations, and law students' annotations. The agreements among these groups support that using GPT-4 as a pre-annotation tool is a useful starting point for large-scale annotation of factors.

In making legal decisions, courts apply relevant law to facts. While the law typically changes slowly over time, facts vary from case to case. Nevertheless, underlying patterns of fact may emerge. This research focuses on underlying fact patterns commonly present in cases where motorists are stopped for a traffic violation and subsequently detained while a police officer conducts a canine sniff of the vehicle for drugs. We present a set of underlying patterns of fact, that is, factors of suspicion, that police and courts apply in determining reasonable suspicion. We demonstrate how these fact patterns can be identified and annotated in legal cases and how these annotations can be employed to fine-tune a transformer model to identify the factors in previously unseen legal opinions.

Probable cause determinations are problematic. Like all court decisions using totality-of-the-circumstances tests, it is difficult to use one decision – or even a few – to foresee a subsequent outcome. No human is capable of reading all the relevant Fourth Amendment opinions relevant to resolving any search and seizure issue. Machines may be capable of this task and to do so they will need to be able to identify particular types of suspicious factors from the various ways courts describe the factors. This project examines the ability of three machine learning models to examine the relevant text of opinions to identify the suspicious factors courts used to determine whether adequate suspicion existed from an intrusion protected by the Fourth Amendment.

Determining if a court has applied a bright-line or totality-of-the-circumstances rule for Fourth Amendment cases demonstrates a difficult problem even for human lawyers and justices. Determining the type of test that governs an issue is essential to answering a legal question. Modern natural language processing (NLP) tools, such as transformers, demonstrate the capacity to extract relevant features from unlabelled text. This study demonstrates the effectiveness of the BERT, RoBERTa, and ALBERT transformer models to classify Fourth Amendment cases by bright-line or totality-of-the-circumstances rule. Two approaches are considered in which models are trained with either positive language extracted by a domain-expert or with full texts of cases. Transformers attain up to 92.31% accuracy on full texts, further demonstrating the capability of NLP techniques on domain-specific tasks even without handcrafted features.

Police officers must daily determine whether they have justification to detain cars they have stopped for ordinary traffic investigations for further investigation. Yet these determinations involve the interpretation of very fact-specific case law that does not yield predictions for subsequent cases and are fraught with subjectivity if not actual bias. Artificially intelligent systems hold the potential to lessen the impact of implicit biases by assisting officers in making these decisions with greater consistency on the basis of factors relevant to suspicion. Using patented text recognition algorithms in order to identify content of interest, or relevant language, our prototype is capable of reading case law and police reports to identify factors relevant to suspicion. With this information, the likelihood a court will approve a search or detention can be assessed. Police reports identifying the bases for fruitful and unsuccessful searches will then permit the system to assess the odds that drugs are present. Deployment will further allow the collection of more detailed data about the basis of successful and unsuccessful stops, improving the system's predictive capacity.