Text classification is a well-established area of machine learning that involves automatically categorizing text into predefined categories, such as positive or negative in sentiment analysis. It typically involves applying a machine learning algorithm to learn a predictive model from a set of labeled training texts and using the model to classify new texts. Large language models (LLMs) have been successfully applied to various natural language processing tasks, including text classification. There are two main approaches to text classification with LLMs: In-Context Learning and Fine-Tuning. In-Context Learning involves prompt engineering, allowing the model to learn a new task using only a few demonstration examples, while Fine-Tuning adjusts the model’s parameters with additional labeled data. Retrieval-Augmented Generation (RAG) is a retrieval process that enhances LLM performance by selecting relevant examples. This paper presents our work on utilizing In-Context Learning and RAG to identify delay-related statements during the document review process of construction delay disputes. We also report the results of our experiments comparing the accuracy of In-Context Learning with that of traditional machine learning algorithms, such as logistic regression and KNN.

The costs involved in manually reviewing documents in legal civil litigations have grown dramatically as more and more information is stored electronically. As a result, the document review process can require an extraordinary dedication of resources. In construction litigations, quickly finding supporting documentation in a delay dispute is critical to the success of a matter. Identifying relevant delay-related communications and supporting documentation has historically been expensive and time consuming. Using machine learning technologies, respondents can be more comprehensive in their assessment of the data requiring review to respond to the claim in time. Explainable machine learning is an active machine learning research area, and in an explainable machine learning system, predictions generated from a machine learning model are explainable and human understandable. In delay dispute ‘document review’ scenarios, a document can be identified as delay-related, as long as one or more of the text snippets in a document are deemed delay-related. In these scenarios, if these delay-related snippets can be located, then attorneys could easily evaluate the model’s decision. The authors of this paper propose an approach for accurately identifying rationales and an approach for boosting document classification accuracy using delay-related snippets and their applications in construction delay disputes. The authors conducted experiments using data from a few real world delay dispute matters and the results from these experiments show that the proposed approaches have the potential to significantly advance the application of text classification in document review in construction delay dispute matters.