The HERITAGE-AI (Harnessing Enhanced Research and Instructional Technologies for Archival Generative Exploration using AI), as part of the IMLS grant initiative, GenAI-4-Archive, aims to analyze sensitive historical datasets ethically using advanced AI technologies. One of the key tasks of this project focuses on selecting the most suitable Large Language Model (LLM) for analyzing the Domestic Traffic Ads (DTA) published in Maryland between 1824 and 1864 by slave traders—a dataset rich in historical significance yet fraught with ethical considerations. Analyzing sensitive historical datasets presents unique ethical and technical challenges. This paper presents a comparative evaluation of leading LLMs to identify the optimal model to meet HERITAGE-AI’s objectives. We survey contemporary models, including OpenAI’s GPT-4o, Anthropic’s Claude Sonnet, Meta’s Llama 3.2, and Google’s Gemini, to identify the most suitable model for Generative AI-based analysis of the DTA dataset. The objective is to select an LLM that can handle the sensitive nature of the data responsibly while providing accurate and insightful analysis. Three critical evaluation criteria, among others, are established for this reason: Sensitivity to Historical Context, Privacy and Security, and Customizability. Our analysis follows a three-step approach: evaluating free versions, paid versions, and enterprise-grade cloud-based implementations of these LLMs. Our findings reveal that while free and paid versions offer varying degrees of accessibility, they fall short in providing the necessary privacy, security, multi-user access, and customization required for analyzing sensitive historical data like the DTA dataset. In the third step, by comparing the cloud-based implementations of Azure OpenAI’s GPT-4o, AWS Bedrock’s Claude, and AWS Bedrock’s Llama3.2 LLMs, Azure openAI GPT-4o emerges as the most suitable option for this project. Although GPT-4o and Claude were close contenders, Gpt-4o demonstrated robust mechanisms due to its high accuracy, ethical sensitivity, robust privacy controls, and scalability in a cloud-based environment. It also offers extensive customizability, allowing for effective integration of the DTA dataset and alignment with the project’s ethical standards. Future work will involve domain experts and community members in implementing Azure OpenAI GPT-4o for the DTA dataset analysis.

This paper examines literature from the field of Computational Archival Science (CAS) to track efforts to address the challenges of the digital age in archives. The Digital Tsunami presented archives with a problem of scale, challenges with Digital Fragility, and changing modes of access which CAS sought to address with computational methods. The born-digital revolution presents the challenge of the Vanishing Box - the loss of topical and temporal structure in records created by the digital workforce leaving collections of Virtual Machines containing chaotic virtual drifts of items with limited metadata suffering from delays in digital preservation. This raised further issues of scale as well as requiring a fundamental rethinking of foundational theories of archival science which CAS sought to address with changes to the Appraisal, Records Management, Description, and Preservation practices previously developed for analog and digitized records.

This paper presents ChatLoS-DTA, a custom Generative Pre-trained Transformer (GPT) model specifically developed for data analysis on the Domestic Traffic Ads (DTA) Legacy of Slavery dataset. The DTA dataset consists of numerous historical newspaper advertisements from 1824 to 1864 for buying and selling enslaved individuals across Maryland. This dataset, digitized and curated by the Maryland archives, offers valuable insights into patterns within the domestic slave trade. However, certain accessibility challenges exist for non-technical users, including the descendants of enslaved individuals or cultural researchers. ChatLoS-DTA, built on OpenAI’s ChatGPT-4 and Python libraries, was designed to allow such users to query the dataset using natural language without requiring any technical expertise. This paper discusses ChatLoS-DTA’s architecture, ethical framework, performance, and limitations, highlighting the model’s potential as a template for applying generative AI in cultural and historical research. Future work includes refining the tool’s accuracy to broaden dataset compatibility and further enhance ethical safeguards.

This paper discusses the use of Computational Thinking (CT) in Archival Educators’ instruction towards enhancing the training and professional development of the library and archival workforce to meet the needs of their communities, and enhancing digital collection management and access to information and resources through retrospective and born-digital content. Four educators share their teaching strategies aimed at modernizing the way digital LIS and computational education are conducted. Their goal is to create an active and engaged community of future archival practitioners, ready to tackle the digital records and archives future.

Entity Resolution (ER) is increasingly being used to identify and link names across archival collections. We describe a framework for unlocking and linking biographical data from WWII Japanese American Incarceration Camps using Entity Resolution and other computational approaches. We demonstrate the construction of social graphs that link people, places, and events and which support further scholarship and reveal hidden stories in historical events, especially given contested archival sources. Finally, we show the power of computational analysis to recreate event networks and represent movement of people using maps. This type of modeling is captured through interactive Jupyter Notebooks that integrate these various elements and document our interpretation of Japanese American experiences and events at the Tule Lake concentration camp.

Understanding handwritten and printed text is easier for humans but computers do not have the same level of accuracy. While there are many Optical Character Recognition (OCR) tools like PyTesseract1, Abbyy FineReader2which extract the text as digital characters from handwritten or printed text images, none of them are without unrecognizable characters or misspelled words. Spelling correction is one of the well-known tasks in Natural Language Processing. Spelling correction of an individual word could be performed through existing tools, however, correcting a word based on the context of the sentence is a challenging task that requires a human-level understanding of the language. In this paper, we introduce a novel experiment of applying Natural Language Processing using a machine learning concept called Transfer Learning3on the text extracted by OCR tools, thereby optimizing the output text by reducing misspelled words. This experiment is conducted on the OCR output of a sample of newspaper images published between the late 18th century to 19th century. These images were obtained from the Maryland State Archives4digital archives project named, the Legacy of Slavery5. This Natural Language Processing approach uses pre-trained language transformer models like BERT6and RoBERTa7which are used as word-prediction software for spelling correction based on the context of the words in the OCR output. We compare the performance of BERT and RoBERTa on two OCR tool outputs, namely PyTesseract and Abbyy FineReader. A comparative evaluation shows that both the models work fairly well on correcting misspelled words considering the irregularities in the text data from the OCR output. Additionally, with the Transfer Learning output text, a special process is conducted to create a new feature that originally did not exist in the original dataset dataset using Spacy’s Entity Recognizer (ER)8. This new extracted values are added to the dataset as a new feature. Also, an existing feature’s values are compared to Spacy’s ER output and the original hand transcribed data.

This paper proposes, tests, and evaluates an innovative Computational Archival Science (CAS) framework to enhance the ability to link people, places, and events depicted in historical photography collections. The protocol combines elements of computer vision with natural language processing, entity extraction, and metadata linking techniques to transform and connect existing archival metadata. Development of the framework is built upon a case study based on the Spelman College Archives Photograph Collection and provides background information, reports on the text processing, image analysis, semantic linking, and evaluation aspects associated with the design and use of the AI-supported framework.

Graduate students at the University of Maryland's College of Information Studies (UMD iSchool) collaborated in interdisciplinary teams on a case study to explore application of computational methodologies to datafied collections related to slavery in the Maryland State Archives (MSA). Two research questions were examined: (1) What are the opportunities and limitations for using computational methods and open source tools to characterize data encoded within records of enslavement and to discover new patterns and relationships in that data? (2) How does knowledge of social and cultural systems impact those opportunities and limitations? Computational methods and tools were most effectively used when socio-cultural contextualization and technology's role as a mediator of representation were taken into account. Three additional technical research areas are identified to enhance recovery of heritage hidden in records of enslavement: visualization, graph databases, and ontologies and metadata.

In this paper, we present a series of experiments we conducted over the summer of 2020 with the FDR Morgenthau Holocaust Collections at the FDR Presidential Library and Museum, in order to unlock hard-to-reach information in the collections and improve access to the public and researchers. We extract detailed Subject Index metadata from Table of Contents images towards creating better finding aids. We demonstrate how digital curation of archival collections are a necessary preparation step for use with supervised Machine Learning algorithms. Finally, we introduce the notion of historical contextualization of Machine Learning models in order to create culturally-aware training models.

We describe the development and testing of the next-generation Trellis Linked Data Platform with Memento versioning support. In addition to highlighting several features that set this system apart from others, we elaborate on the extensive testing and compatibility work that was done in order to align this system with the Fedora 5.0 specification. We draw attention to the performance and scaling features provided by the Trellis Linked Data Platform in general and by the Cassandra database back end. We review the profound impact that such a system can have on demanding, next generation use cases, such as crowd sourcing, machine learning, and direct file access by desktop applications.

We describe the value of reframing digital curation practices through a computational thinking (CT) framework. Using a case study that demonstrates computational treatments of World War II Japanese-American Incarceration Camp Records, we demonstrate the applicability of CT with respect to: (1) Detecting personally identifiable information, (2) Developing name registries, (3) Integrating vital records, (4) Designing controlled vocabularies, (5) Mapping events and people, and (6) Connecting events and people through networks. The work was carried out by 5 teams of students in an 8week digital curation exploration and development sprint.

This paper explores whether the computational thinking practices of mathematicians and scientists in the physical and biological sciences are also the practices of archival scientists. It is argued that these practices are essential elements of an archival science education in preparing students for a professional archival career.

The Maryland State Archives (MSA) and the Digital Curation Innovation Center (DCIC) of the University of Maryland's iSchool are collaborating on a digital project that utilizes digital strategies and technologies to create an in-depth understanding of the African-American experience in Maryland during the era of slavery. Utilizing crowdsourcing for transcription, data cleaning and transformation techniques, and data visualization strategies, the joint project team is creating new avenues for understanding the complex web of relationships that undergirded the institution of slavery. iSchool students, full participants on the project team, are learning digital curation and other technical skills while gaining insights into the multiple uses of how cultural Big Data can penetrate the past and illuminate the present.

We describe computational treatments of archival collections through a case study involving World War II Japanese-American Incarceration Camps. We focus on automating the detection of personally identifiable information or PII. The paper also discusses the emergence of computational archival science (CAS) and the development of a computational framework for library and archival education. Computational Thinking practices are applied to Archival Science practices. These include: (1) data creation, manipulation, analysis, and visualization (2) designing and constructing computational models, and (3) computer programming, developing modular computational solutions, and troubleshooting and debugging. We conclude with PII algorithm accuracy, transparency, and performance considerations and future developments.

The discipline of professional archivists is rapidly changing. Most contemporary records are created, stored, maintained, used and preserved in digital form. Most graduate programs and continuing education programs in Archival Studies address this challenge by introducing students to information technology as it relates to digital records. We propose an approach to addressing this challenge based on introducing computational thinking into the graduate archival studies curriculum.

Computational Archival Science (CAS) has been proposed as a trans-disciplinary field that combines computational and archival thinking. To provide grounded evidence, a foundational paper explored eight initial themes that constitute potential building blocks [1]. In order for a CAS community to emerge, further studies are needed to test this framework. While the foundational paper for CAS provides a conceptual and theoretical basis of this new field, there is still a need to articulate useful guidelines and checkpoints that validate a CAS research agenda. In this position paper, we propose heuristics for assessing emerging CAS-related studies that researchers from traditional fields can use in their research design stage. The Human Face of Big Data project, a digital curation and interface design project for urban renewal data, is presented and analyzed to demonstrate the validity of the suggested heuristics. Finally, implications for CAS and future work are discussed.

This paper describes the linguistic analysis of index note cards from record series of the World War II Japanese-American Internment Camps that are in the custody of the National Archives. It also describes the use of GATE Developer, and an extension of ANNIE, a GATE plugin, in linguistic processing of information specific to index note cards in order to extract metadata supporting access and archival decisions regarding record release and withdrawal. The content of the index cards will be interpreted as OWL/RDF statements. Those statements will be stored in a graph database and used with objects such as digital maps and photos to produce an interactive user interface to exhibit events at relocation centers.

This paper discusses a project that studied the relationship between citizen trust and social protest using visual analysis of approximately 11 million sentiment classified Tweets from the period of the 2014 Brazilian World Cup. The results of the study reveal that the 2014 World Cup protests in Brazil sprang from a wide range of grievances coupled with a relative sense of deprivation compared with emergent comparative `standards'. This sense of grievance gave rise to sentiments that activated online protest that may have led to other forms of social protest, such as demonstrations. The paper describes an innovative approach to big data analytics-mixed initiative social media analytics - and discusses the potential of using big data in social science research of this kind, as well as some of the open methodological, technical and ethical issues still to be addressed.

We present Brown Dog, two highly extensible services that aim to leverage any existing pieces of code, libraries, services, or standalone software (past or present) towards providing users with a simple to use and programmable means of automated aid in the curation and indexing of distributed collections of uncurated and/or unstructured data. Data collections such as these encompassing large varieties of data, in addition to large amounts of data, pose a significant challenge within modern day "Big Data" efforts. The two services, the Data Access Proxy (DAP) and the Data Tilling Service (DTS), focusing on format conversions and content based analysis/extraction respectively, wrap relevant conversion and extraction operations within arbitrary software, manages their deployment in an elastic manner, and manages job execution from behind a deliberately compact REST API. We describe both the motivation and need/scientific drivers for such services, the constituent components that allow for arbitrary software/code to be used and managed, and lastly an evaluation of the systems capabilities and scalability.

This short paper discusses the “Integrating Data Mining and Data Management Technologies for Scholarly Inquiry” project. In this “Round Two” Digging Into Data Challenge award, we explored uses and approaches for large-scale data analysis and processing for the Humanities and Social Sciences through the integration of several infrastructure frameworks: Cheshire, iRODS, and Amazon Web Services (EC2 computing and S3 storage). Our “big data” consisted of the entire texts collection of the Internet Archive (approximately 3.6 million volumes) and the entire JSTOR database. We performed surface-level natural language processing on this data to identify noun phrases and further refinements to identify personal, corporate, and geographic names. We then used resources including library and archival authority records to identify variants and merge names. The goal is to create an integrated index of persons, places, and organizations referenced in our collections.

The Cyber-Infrastructure for Billions of Electronic Records (CI-BER) project is a collaborative big data management project based on the integration of heterogeneous datasets and multi-source historical and digital collections, including a place-based citizen-led crowdsourcing case study of the Southside neighborhood in Asheville, North Carolina. The project is funded by the National Science Foundation and the National Archives and Records Administration (NARA) agencies. A test-bed collection containing nearly 100 million files and 50TB of data was developed, with content representing electronic Federal Government records from 150 federal agencies. The CI-BER project advances the state of the art in generalizable and extensible ultra-highly scalable data management architectures, potentially enabling robust technical preservation of, and access to, electronic records and digital data in the context of emerging national scale cyber-environments. A first-generation open source collaborative mapping environment prototype is currently being developed to support novel “citizen-led crowdsourcing” possibilities for archival material.

The MIX mediator system, MIXm, is developed as part of the MIX Project at the San Diego Supercomputer Center, and the University of California, San Diego.1 MIXm uses XML as the common model for data exchange. Mediator views are expressed in XMAS (XML Matching And Structuring Language), a declarative XML query language. To facilitate user-friendly query formulation and for optimization purposes, MIXm employs XML DTDs as a structural description (in effect, a “schema”) of the exchanged data. The novel features of the system include:

Large spatial interpolation problems present significant computational challenges even for the fastest workstations. In this paper we demonstrate how parallel processing can be used to reduce computation times to levels that are suitable for interactive interpolation analyses of large spatial databases. Though the approach developed in this paper can be used with a wide variety of interpolation algorithms, we specifically contrast the results obtained from a global ‘brute force’ inverse–distance weighted interpolation algorithm with those obtained using a much more efficient local approach. The parallel versions of both implementations are superior to their sequential counterparts. However, the local version of the parallel algorithm provides the best overall performance.

Statistical measures of spatial association have significant computational requirements when large data sets are analysed. In this paper, a measure of spatial association, G(d), is used to illustrate how a massively parallel computer can be used to address the computational requirements of spatial statistical analysis. The statistical algorithms were implemented using two MasPar MP-1 computers, one with 8192 and the other with 16834 processors, and an MP-2 machine with 4096 processors. The results demonstrate that substantial reductions in processing times can be achieved using massively parallel architectures: when compared to a superscalar workstation, speed-up values in excess of 20 were obtained. The design of parallel programmes, however, requires careful planning since many factors under programmer control affect the efficiency of the resulting computations.