The task of forecasting case importance has received far less attention in the legal domain compared to judgment prediction, but it is a task that is an essential step to capture within tools for assisting with the processing of cases submitted to a court. In this paper we propose a cornerstone framework for carrying out the task of forecasting case importance, using communicated cases, which are documents available prior to any decision being issued. The setting for our work is cases in the European Court of Human Rights, with a specific focus on Article 3, prohibition of torture. We set out proposals for a Retrieval-Augmented Generation (RAG) framework that makes use of Large Language Models augmented with Semantic Search, Knowledge Graph and Re-Ranking components and we evaluate the effectiveness of this framework and its components. Further experiments conducted evaluate the framework using both pre-trained and fine-tuned LLMs, as well as use of different prompting strategies. The basic experiments show a propensity for the LLMs to significantly overestimate the importance of cases, but when we augment the LLMs with the aforementioned components, we are able to gain uplifts in performance. Our framework and results provide a solid basis for determining the requirements for the development of successful automated tools to be used to assist with determining case importance.

We report on a first set of results from experiments undertaken to tackle the novel task of determining the optimal context window for extracting and contextualising citation instances within case law. The wider task of outcome prediction using AI tools cannot be undertaken without considering the role that citations play when new cases are being decided. This short paper aims to shine a light on the importance of this task and provide the foundation for developing AI tools to capture citations’ context by examining a corpus of legal cases taken from the European Court of Human Rights. Our results show that there is an identifiable “Goldilocks Zone” of scoped paragraph-level context windows that attention can be focused on for extracting citation instances.

We present a context-aware approach to citation analysis for European Court of Human Rights case law. Instead of a single edge between cases, we annotate each citation instance with respect to its individual context. We construct two datasets: a new human-annotated set spanning judgments and decisions across Grand Chamber, Chamber, and Committee, and an AI-annotated set of judgments produced at scale. We find that most authorities are cited at least twice within a case, and repeated citations frequently vary by complaint and judicial consideration. Our empirical analysis shows that AI annotations are highly competitive with outputs from trained human annotators, but with significant variation depending on the provision of convention and the level of the Court producing the ruling. We release the datasets and an interactive Citation Analysis Tool that enables context-filtered retrieval, supporting triage and research grounded in past precedent.

Online reviews now have a considerable influence on consumer choices. However, little work has focused on what features of review platforms influence review quality. We present a novel approach to identify the features that encourage quality reviews. By interpreting reviews as arguments for or against the product, an argument scheme can be used to simulate the emergent reliability of reviews resulting from different setups of the online review platform. Our results show that if the most recent, helpful, or polarised reviews are promoted over quality, then good quality reviews will almost never be shown to users.

Large Language Models (LLMs) have become a significant milestone in the history of artificial intelligence, representing a powerful technology that drives advancements in natural language understanding and generation. In this paper, we propose an approach in which LLMs are utilized to support the task of translating natural language arguments into computational representations. Our approach is grounded in using argumentation schemes to classify arguments, providing context to LLMs for performing the proposed task. Our results demonstrate that LLMs, even with a short context, can handle simple argument structures. Moreover, our findings suggest that a larger context would likely enhance the performance, particularly when dealing with more complex argument structures.

We report on a study undertaken to analyse AI performance on two tasks involved in automating processing of cases from the European Court of Human Rights: classification of legal case outcomes and keyword prediction. Results show variation across Articles and Court levels, and challenge the common viewpoint that larger legal corpora combined with larger models will be sufficient for effective automated legal reasoning. Legal summarisation, as reflected with keyword prediction, proved more challenging than outcome classification. Our results suggest the need for improved case law retrieval and understanding of contextual factors for effective automated legal decision support.

Recent years have witnessed significant progress in the deployment of advanced Natural Language Processing (NLP) techniques based on transformer technology, across many domains and applications. However, in legal domains, due to the complexity, length, and sparsity of legal case documents, the use of these advanced NLP techniques has offered comparatively slight returns. Perhaps even more importantly, such methods are critically lacking in explainability and justification of outputs, which are essential for many legal applications. We propose that the direction of these NLP techniques should be aimed at ascription to a legal knowledge model, which can then provide the necessary and auditable justifications for the rationale of any case outcome. In this paper we investigate the effectiveness of using Hierarchical Bidirectional Encoder Representations from Transformers (H-BERT) models to ascribe to an Angelic Domain Model (ADM) that is able to represent the legal knowledge of a domain in a structured way, enabling justifications and improving performance. Our study involved an annotation task on a popular domain, cases from the European Court of Human Rights, to gain an understanding of the balance of complaints in the domain. The data set produced from this study enabled training of models for factor ascription using the classification targets derived from the annotations. We present results of experiments conducted to evaluate the performance of the ascription task at three different levels of abstraction within the structured model.

We report a study undertaken to analyse human performance on the verdict classification task. Several approaches have addressed this task with outcomes compared against the outcomes from actual legal cases. Results vary and we investigate how classification is done by humans. A key finding is that fact descriptions alone are insufficient for accurate classification, independent of legal background.

Typically in abstract argumentation, one starts with arguments and a defeat relation, and applies some semantics in order to determine the acceptability status of the arguments. We consider the converse case where we have knowledge of the acceptability status of arguments and want to identify a defeat relation that is consistent with the known acceptability data – the σ-consistency problem. Focusing on complete semantics as underpinning the majority of the major semantic types, we show that the complexity of determining a defeat relation that is consistent with some set of acceptability data is highly dependent on how the data is labelled. The extension-based 2-valued σ-consistency problem for complete semantics is revealed as NP-complete, whereas the labelling-based 3-valued σ-consistency problem is solvable within polynomial time. We then present an informal discussion on application to grounded, stable, and preferred semantics.

Reasoning with legal cases has long been modelled using symbolic methods. In recent years, the increased availability of legal data together with improved machine learning techniques has led to an explosion of interest in data-driven methods being applied to the problem of predicting outcomes of legal cases. Although encouraging results have been reported, they are unable to justify the outcomes produced in satisfactory legal terms and do not exploit the structure inherent within legal domains; in particular, with respect to the issues and factors relevant to the decision. In this paper we present the technical foundations of a novel hybrid approach to reasoning with legal cases, using Abstract Dialectical Frameworks (ADFs) in conjunction with hierarchical BERT. ADFs are used to represent the legal knowledge of a domain in a structured way to enable justifications and improve performance. The machine learning is targeted at the task of factor ascription; once factors present in a case are ascribed, the outcome follows from reasoning over the ADF. To realise this hybrid approach, we present a new hybrid system to enable factor ascription, envisioned for use in legal domains, such as the European Convention on Human Rights that is used frequently in modelling experiments.

Explanation and justification of legal decisions has become a highly relevant topic in light of the explosion of interest in the use of machine learning (ML) approaches to predict legal decisions. Current suggestions are to use the established factor based explanations developed in AI and Law as the basis for explaining such programs. We, however, identify factor ascription as an important aspect of explanation of case outcomes not currently covered, and argue that explanations must also include this aspect. Finally, we outline our proposal for a hybrid system approach that combines ML and Abstract Dialectical Framework (ADF) layers to engender an explainable process.