This work presents a computational approach to analyze character development along the narrative timeline.The analysis characterizes changes in the protagonist's views and behavior and the interplay between them.We consider transcripts of Holocaust survivor testimonies as a test case, each telling the story of an individual in first-person terms.We focus on the survivor's religious trajectory, examining the evolution of their disposition toward religious belief and practice as it is reflected in the testimony.Clustering the resulting trajectories in the dataset, we identify common sequences in the data.Our findings highlight multiple common structures of religiosity across the narratives: in terms of belief, a constant disposition is common, while for practice, most present an oscillating structure, serving as valuable material for historical and sociological research.This work demonstrates the potential of natural language processing for analyzing character evolution through thematic trajectories in narratives.

Learning a model of a stochastic setting often involves learning both general structure rules and specific properties of the instance.This paper investigates the interplay between learning the general and the specific in various learning methods, with emphasis on sample efficiency.We design a framework called LEV-ERWORLDS, which allows the generation of simple physics-inspired worlds that follow a similar generative process with different distributions, and their instances can be expressed in natural language.These worlds allow for controlled experiments to assess the sample complexity of different learning methods.We experiment with classic learning algorithms as well as Transformer language models, both with fine-tuning and In-Context Learning (ICL).Our general finding is that (1) Transformers generally succeed in the task; but (2) they are considerably less sample efficient than classic methods that make stronger assumptions about the structure, such as Maximum Likelihood Estimation and Logistic Regression.This finding is in tension with the recent tendency to use Transformers as general-purpose estimators.We propose an approach that leverages the ICL capabilities of contemporary language models to apply simple algorithms for this type of data.Our experiments show that models currently struggle with the task but show promising potential.1

Although language model scores are often treated as probabilities, their reliability as probability estimators has mainly been studied through calibration, overlooking other aspects.In particular, it is unclear whether language models produce the same value for different ways of assigning joint probabilities to word spans.Our work introduces a novel framework, ConTestS (Consistency Testing over Spans), involving statistical tests to assess score consistency across interchangeable completion and conditioning orders.We conduct experiments on post-release real and synthetic data to eliminate training effects.Our findings reveal that both Masked Language Models (MLMs) and autoregressive models exhibit inconsistent predictions, with autoregressive models showing larger discrepancies.Larger MLMs tend to produce more consistent predictions, while autoregressive models show the opposite trend.Moreover, for both model types, prediction entropies offer insights into the true word span likelihood and therefore can aid in selecting optimal decoding strategies.The inconsistencies revealed by our analysis, as well their connection to prediction entropies and differences between model types, can serve as useful guides for future research on addressing these limitations.1

This work focuses on the spatial dimension of narrative understanding and presents the task of event-location tracking in narrative texts, namely the extraction of the sequence of locations where the narrative is set.We present several architectures for the task that seek to model the global structure of the sequence, with varying levels of context awareness.We compare these methods to a number of strong baselines and ablated variants.We also develop methods for the generation of location embeddings and show that learning to predict a sequence of continuous embeddings is advantageous in terms of performance over predicting a string of locations.We focus on the test case of Holocaust survivor testimonies, motivated by the moral and historical importance of studying this dataset using computational means.The dataset further provides a unique case of a large set of narratives with a relatively restricted set of location trajectories.Our results show that models that are aware of the global context of the narrative can generate more accurate location chains.We corroborate the effectiveness of our methods by showing similar trends in an additional domain. 1

The task of topical segmentation is well studied, but previous work has mostly addressed it in the context of structured, well-defined segments, such as segmentation into paragraphs, chapters, or segmenting text that originated from multiple sources.We tackle the task of segmenting running (spoken) narratives, which poses hitherto unaddressed challenges.As a test case, we address Holocaust survivor testimonies, given in English.Other than the importance of studying these testimonies for Holocaust research, we argue that they provide an interesting test case for topical segmentation, due to their unstructured surface level, relative abundance (tens of thousands of such testimonies were collected), and the relatively confined domain that they cover.We hypothesize that boundary points between segments correspond to low mutual information between the sentences proceeding and following the boundary.Based on this hypothesis, we explore a range of algorithmic approaches to the task, building on previous work on segmentation that uses generative Bayesian modeling and state-of-the-art neural machinery.Compared to manually annotated references, we find that the developed approaches show considerable improvements over previous work.1