This paper presents a comparative study evaluating the performance of Large Language Models (LLMs) in generating timeline summaries from construction delay documents. We assessed seven open-source LLMs and two commercial chatbots (ChatGPT and Claude) on their ability to extract, organize, and summarize delay events from twenty-one carefully curated synthetic snippets of text. The evaluation framework combined automatic metrics (BERTScore and ROUGE scores) with expert human assessment across four dimensions: event description accuracy, date accuracy, event capture completeness, and language quality.Results demonstrate that while commercial solutions, particularly Claude, achieved superior performance, several open-source alternatives showed comparable capabilities. Notably, Llama-3.1-70B-Instruct showed robust performance in event capture and source tracking, while Llama-3.1-8B-Instruct offered efficient processing with balanced performance among smaller models. A critical finding was the widespread challenge in temporal information processing, with only Claude achieving complete accuracy in date extraction and event association. The study's findings suggest that open-source LLMs can serve as practical tools for construction document analysis, although model selection is a critical consideration based on specific accuracy and efficiency requirements, and resource constraints.

Text embeddings are numerical representations of text data, where words, phrases, or entire documents are converted into vectors of real numbers. These embeddings capture semantic meanings and relationships between text elements in a continuous vector space. The primary goal of text embeddings is to enable the processing of text data by machine learning models, which require numerical input. Numerous embedding models have been developed for various applications. This paper presents our work in evaluating different embeddings through a comprehensive comparative analysis of four distinct models, focusing on their text classification efficacy. We employ both K-Nearest Neighbors (KNN) and Logistic Regression (LR) to perform binary classification tasks, specifically determining whether a text snippet is associated with 'delay' or 'not delay' within a labeled dataset. Our research explores the use of text snippet embeddings for training supervised text classification models to identify delay-related statements during the document review process of construction delay disputes. The results of this study highlight the potential of embedding models to enhance the efficiency and accuracy of document analysis in legal contexts, paving the way for more informed decision-making in complex investigative scenarios.

In the realm of U.S. legal practice, safeguarding privileged communications and data from unintended disclosure is of utmost importance. Traditionally, attorneys have relied on keyword searches and manual reviews to find privileged documents in legal cases. However, as data volumes grow, this method becomes increasingly costly and inefficient. Machine learning techniques have emerged as a viable solution for identifying privileged documents. This paper introduces a hybrid approach to identifying privileged documents by integrating rule-based modeling logistic regression, and active learning within a predictive analytics framework. This method streamlines privilege prediction in legal datasets by leveraging metadata, such as attorney and firm details, along with document text, to create a comprehensive end-to-end process for privilege identification. Evaluation through precision and recall metrics on the dataset from a real-world legal matter demonstrates the effectiveness of this approach in classifying privileged status in complex legal documents, providing a scalable and accurate solution for automated document review in legal contexts.

Corporate litigation costs associated with manually reviewing documents in legal matters continue to rise, driven by the ever-expanding volumes of data generated by businesses. To address this challenge, legal teams have increasingly turned to text classification techniques, commonly referred to in the legal community as predictive modeling, predictive coding, or "Technology Assisted Review (TAR)", to identify relevant documents more efficiently during the discovery phase of legal matters. By streamlining the classification process, predictive coding has proven instrumental in minimizing both the time and cost associated with legal discovery. As more corporations expand their operations across global regions, legal matters increasingly involve multilingual datasets, introducing significant complexities for the application of predictive coding. Traditional predictive models exhibit bias towards the dominant language within the model’s training data. The prevalence of multiple languages within the same document further complicates the challenge. This can lead to inefficiencies, elevated costs, and risks of inaccurate classifications of nondominant language documents.This research empirically examines two methods for applying predictive coding to multilingual datasets: a single multilingual model approach and a language-specific modeling approach. This group of collaborators posits that language-specific models will outperform the single multilanguage model approach in evaluating document relevance. The research used a multilingual dataset from a recent confidential real-world legal matter containing predominantly English and Chinese document text Performance was assessed using precision and recall measures. Our results show that the ensemble approach – employing language-specific models for classifying documents in their respective languages and employing the mixed language model across all remaining documents—outperforms a single multilanguage model. The outcomes of this research aim to inform the workflow considerations for legal professionals undertaking multilingual discovery efforts. The findings will help legal practitioners make data-driven decisions on the most efficient and accurate modeling approach for multilingual document review, shaping the future of predictive coding in cross-linguistic e-discovery.

Protecting privileged communications and data from disclosure is paramount for legal teams. Unrestricted legal advice, such as attorney-client communications or litigation strategy. are vital to the legal process and are exempt from disclosure in litigations or regulatory events. To protect this information from being disclosed, companies and outside counsel must review vast amounts of documents to determine those that contain privileged material. This process is extremely costly and time consuming. As data volumes increase, legal counsel employ methods to reduce the number of documents requiring review while balancing the need to ensure the protection of privileged information. Keyword searching is relied upon as a method to target privileged information and reduce document review populations. Keyword searches are effective at casting a wide net but return over inclusive results - most of which do not contain privileged information - and without detailed knowledge of the data, keyword lists cannot be crafted to find all privilege material. Overly-inclusive keyword searching can also be problematic, because even while it drives up costs, it also can cast `too far of a net' and thus produce unreliable results.To overcome these weaknesses of keyword searching, legal teams are using a new method to target privileged information called predictive modeling. Predictive modeling can successfully identify privileged material but little research has been published to confirm its effectiveness when compared to keyword searching. This paper summarizes a study of the effectiveness of keyword searching and predictive modeling when applied to real- world data. With this study, this group of collaborators wanted to examine and understand the benefits and weaknesses of both approaches to legal teams with identifying privilege material in document populations.

Training documents have a significant impact on the performance of predictive models in the legal domain. Yet, there is limited research that explores the effectiveness of the training document selection strategy - in particular, the strategy used to select the seed set, or the set of documents an attorney reviews first to establish an initial model. Since there is limited research on this important component of predictive coding, the authors of this paper set out to identify strategies that consistently perform well. Our research demonstrated that the seed set selection strategy can have a significant impact on the precision of a predictive model. Enabling attorneys with the results of this study will allow them to initiate the most effective predictive modeling process to comb through the terabytes of data typically present in modern litigation. This study used documents from four actual legal cases to evaluate eight different seed set selection strategies. Attorneys can use the results contained within this paper to enhance their approach to predictive coding.