The widespread application of uninterpretable machine learning systems for sensitive purposes has spurred research into elucidating the decision-making process of these systems. These efforts have their background in many different disciplines, one of which is the field of AI & law. In particular, recent works have observed that machine learning training data can be interpreted as legal cases. Under this interpretation, the formalism developed to study case law, called the theory of precedential constraint, can be used to analyze the way in which machine learning systems draw on training data—or should draw on them—to make decisions. In the present work, we advance the theory underlying these explanation methods, by relating it to order theory and logic. This allows us to write a software implementation of the theory that can be used to compute with the definitions and give automatic proofs of the properties of the model. We use this implementation to evaluate the model on a series of datasets. Through this analysis, we characterize the types of datasets that are more, or less, suitable to be described by the theory.

Case-based reasoning is a central theme of ai and law research, providing formal models to assess case-base consistency. We present four theorems describing how statistical modeling techniques, feature modifications, and data binning affect case-base consistency. Primarily, our findings demonstrate that generalized linear models necessarily yield consistent decisions. We further show that adding input features increases consistency, while removing features decreases it; similarly, output binning increases consistency, whereas input binning decreases it. Each of these theorems is showcased through a consistency analysis of the compas program—a widely used recidivism prediction tool which has been at the center of debates on fairness and interpretability.

In recent years, hierarchical case-based-reasoning models of precedential constraint have been proposed. Trevor Bench-Capon criticised these models, among other things, on the grounds that they would not account for the possibility that intermediate factors are established with different strengths by different base-level factors. In this paper we respond to these criticisms for van Woerkom’s result-based hierarchical models. We argue that in some examples Bench-Capon seems to interpret intermediate factors as dimensions, and that applying van Woerkom’s dimension-based version of the hierarchical result model to these examples avoids Bench-Capon’s criticisms.

In this paper we build on a formal model of reasoning with dimensions to analyze data from the COMPAS program—a widely used and studied tool for predicting recidivism. We extend the underlying theory of the model by introducing a notion of consistency and apply it to assess whether COMPAS follows this principle in its risk assessments and supervision level recommendations. Our analysis yields three key findings. First, the program’s risk score assignments appear highly inconsistent, but we argue this is due to important input features missing from the dataset. Second, the program’s recommended supervision levels do exhibit a high degree of consistency. Third, we uncover errors in the dataset related to the conversion of raw scores to decile scores. These findings cast doubts on previous studies conducted on the COMPAS dataset, and demonstrate the need for evaluation studies like ours.

The widespread application of uninterpretable machine learning systems for sensitive purposes has spurred research into elucidating the decision-making process of these systems. These efforts have their background in many different disciplines, one of which is the field of AI & law. In particular, recent works have observed that machine learning training data can be interpreted as legal cases. Under this interpretation, the formalism developed to study case law, called the theory of precedential constraint, can be used to analyze the way in which machine learning systems draw on training data—or should draw on them—to make decisions. In the present work, we advance the theory underlying these explanation methods, by relating it to order theory and logic. This allows us to write a software implementation of the theory that can be used to compute with the definitions and give automatic proofs of the properties of the model. We use this implementation to evaluate the model on a series of datasets. Through this analysis, we characterize the types of datasets that are more, or less, suitable to be described by the theory.

In recent work, theories of case-based legal reasoning have been applied to the development of explainable artificial intelligence methods, through the analogy of training examples as previously decided cases. One such theory is that of precedential constraint. A downside of this theory with respect to this application is that it performs single-step reasoning, moving directly from the case base to an outcome. For this reason we propose a generalization of the theory of precedential constraint which allows multi-step reasoning, moving from the case base through a series of intermediate legal concepts before arriving at an outcome. Our generalization revolves around the notion of factor hierarchy, so we call this hierarchical precedential constraint. We present the theory, demonstrate its applicability to case-based legal reasoning, and perform a preliminary analysis of its theoretical properties.

In recent years, a model of a fortiori argumentation, developed to describe legal reasoning based on precedent, has been successfully applied in the field of artificial intelligence to improve interpretability of data-driven decision systems. In order to make this model more broadly applicable for this purpose, work has been done to expand the knowledge representation on the basis of which it functions, as the original model accommodates only binary propositional information. In particular, two separate expansions of the original model emerged; one which accounts for non-binary input information, and a second which accommodates hierarchically structured reasoning. In the present work we unify these expansions to a single model, incorporating both dimensional and hierarchical information.