The HERITAGE-AI (Harnessing Enhanced Research and Instructional Technologies for Archival Generative Exploration using AI), as part of the IMLS grant initiative, GenAI-4-Archive, aims to analyze sensitive historical datasets ethically using advanced AI technologies. One of the key tasks of this project focuses on selecting the most suitable Large Language Model (LLM) for analyzing the Domestic Traffic Ads (DTA) published in Maryland between 1824 and 1864 by slave traders—a dataset rich in historical significance yet fraught with ethical considerations. Analyzing sensitive historical datasets presents unique ethical and technical challenges. This paper presents a comparative evaluation of leading LLMs to identify the optimal model to meet HERITAGE-AI’s objectives. We survey contemporary models, including OpenAI’s GPT-4o, Anthropic’s Claude Sonnet, Meta’s Llama 3.2, and Google’s Gemini, to identify the most suitable model for Generative AI-based analysis of the DTA dataset. The objective is to select an LLM that can handle the sensitive nature of the data responsibly while providing accurate and insightful analysis. Three critical evaluation criteria, among others, are established for this reason: Sensitivity to Historical Context, Privacy and Security, and Customizability. Our analysis follows a three-step approach: evaluating free versions, paid versions, and enterprise-grade cloud-based implementations of these LLMs. Our findings reveal that while free and paid versions offer varying degrees of accessibility, they fall short in providing the necessary privacy, security, multi-user access, and customization required for analyzing sensitive historical data like the DTA dataset. In the third step, by comparing the cloud-based implementations of Azure OpenAI’s GPT-4o, AWS Bedrock’s Claude, and AWS Bedrock’s Llama3.2 LLMs, Azure openAI GPT-4o emerges as the most suitable option for this project. Although GPT-4o and Claude were close contenders, Gpt-4o demonstrated robust mechanisms due to its high accuracy, ethical sensitivity, robust privacy controls, and scalability in a cloud-based environment. It also offers extensive customizability, allowing for effective integration of the DTA dataset and alignment with the project’s ethical standards. Future work will involve domain experts and community members in implementing Azure OpenAI GPT-4o for the DTA dataset analysis.

This paper presents ChatLoS-DTA, a custom Generative Pre-trained Transformer (GPT) model specifically developed for data analysis on the Domestic Traffic Ads (DTA) Legacy of Slavery dataset. The DTA dataset consists of numerous historical newspaper advertisements from 1824 to 1864 for buying and selling enslaved individuals across Maryland. This dataset, digitized and curated by the Maryland archives, offers valuable insights into patterns within the domestic slave trade. However, certain accessibility challenges exist for non-technical users, including the descendants of enslaved individuals or cultural researchers. ChatLoS-DTA, built on OpenAI’s ChatGPT-4 and Python libraries, was designed to allow such users to query the dataset using natural language without requiring any technical expertise. This paper discusses ChatLoS-DTA’s architecture, ethical framework, performance, and limitations, highlighting the model’s potential as a template for applying generative AI in cultural and historical research. Future work includes refining the tool’s accuracy to broaden dataset compatibility and further enhance ethical safeguards.

Understanding handwritten and printed text is easier for humans but computers do not have the same level of accuracy. While there are many Optical Character Recognition (OCR) tools like PyTesseract1, Abbyy FineReader2which extract the text as digital characters from handwritten or printed text images, none of them are without unrecognizable characters or misspelled words. Spelling correction is one of the well-known tasks in Natural Language Processing. Spelling correction of an individual word could be performed through existing tools, however, correcting a word based on the context of the sentence is a challenging task that requires a human-level understanding of the language. In this paper, we introduce a novel experiment of applying Natural Language Processing using a machine learning concept called Transfer Learning3on the text extracted by OCR tools, thereby optimizing the output text by reducing misspelled words. This experiment is conducted on the OCR output of a sample of newspaper images published between the late 18th century to 19th century. These images were obtained from the Maryland State Archives4digital archives project named, the Legacy of Slavery5. This Natural Language Processing approach uses pre-trained language transformer models like BERT6and RoBERTa7which are used as word-prediction software for spelling correction based on the context of the words in the OCR output. We compare the performance of BERT and RoBERTa on two OCR tool outputs, namely PyTesseract and Abbyy FineReader. A comparative evaluation shows that both the models work fairly well on correcting misspelled words considering the irregularities in the text data from the OCR output. Additionally, with the Transfer Learning output text, a special process is conducted to create a new feature that originally did not exist in the original dataset dataset using Spacy’s Entity Recognizer (ER)8. This new extracted values are added to the dataset as a new feature. Also, an existing feature’s values are compared to Spacy’s ER output and the original hand transcribed data.

Graduate students at the University of Maryland's College of Information Studies (UMD iSchool) collaborated in interdisciplinary teams on a case study to explore application of computational methodologies to datafied collections related to slavery in the Maryland State Archives (MSA). Two research questions were examined: (1) What are the opportunities and limitations for using computational methods and open source tools to characterize data encoded within records of enslavement and to discover new patterns and relationships in that data? (2) How does knowledge of social and cultural systems impact those opportunities and limitations? Computational methods and tools were most effectively used when socio-cultural contextualization and technology's role as a mediator of representation were taken into account. Three additional technical research areas are identified to enhance recovery of heritage hidden in records of enslavement: visualization, graph databases, and ontologies and metadata.