Recent theoretical advancement of information density in natural language have raised the following question: To what degree does natural language exhibit periodicity pattern in its encoded information?We address this question by introducing a new method called AutoPeriod of Surprisal (APS).APS adopts a canonical periodicity detection algorithm and is able to identify any significant periods that exist in the surprisal sequence of a single document.By applying the algorithm to a set of corpora, we have obtained the following empirical results: Firstly, a considerable proportion of human language demonstrates a strong pattern of periodicity in information.Secondly, new periods that are outside the distributions of typical structural units in text (e.g., sentence boundaries, elementary discourse units, etc.) are found and further confirmed via harmonic regression modeling.We conclude that the periodicity of information in language is a joint outcome from both structured factors and other driving factors that take effect at longer distances.The advantages of our periodicity detection method and its potentials in LLM-generation detection are further discussed.Our code is available as public repositories.1

Backchannels and fillers are important linguistic expressions in dialogue, but often treated as 'noise' to be bypassed in modern transformerbased language models (LMs).Here, we study how they are represented in LMs using three fine-tuning strategies on three dialogue corpora in English and Japanese, in which backchannels and fillers are both preserved and annotated.This allows us to investigate how fine-tuning can help LMs learn these representations.We first apply clustering analysis to the learnt representation of backchannels and fillers, and find increased silhouette scores in representations from fine-tuned models, which suggests that fine-tuning enables LMs to distinguish the nuanced semantic variation in different backchannel and filler use.We also employ natural language generation metrics and qualitative analyses to verify that utterances produced by fine-tuned LMs resemble those produced by humans more closely.Our findings suggest the potential for transforming general LMs into conversational LMs that can produce human-like language more adequately.

Human and model-generated texts can be distinguished by examining the magnitude of likelihood in language.However, it is becoming increasingly difficult as language model's capabilities of generating human-like texts keep evolving.This study provides a new perspective by using the relative likelihood values instead of absolute ones, and extracting useful features from the spectrum-view of likelihood for the human-model text detection task.We propose a detection procedure with two classification methods, supervised and heuristic-based, respectively, which results in competitive performances with previous zero-shot detection methods and a new state-of-the-art on short-text detection.Our method can also reveal subtle differences between human and model languages, which find theoretical roots in psycholinguistics studies.

Linguistic alignment between dialogue partners has been claimed to be affected by their relative social power.A common finding has been that interlocutors of higher power tend to receive more alignment than those of lower power.However, these studies overlook some low-level linguistic features that can also affect alignment, which casts doubts on these findings.This work characterizes the effect of power on alignment with logistic regression models in two datasets, finding that the effect vanishes or is reversed after controlling for low-level features such as utterance length.Thus, linguistic alignment is explained better by low-level features than by social power.We argue that a wider range of factors, especially cognitive factors, need to be taken into account for future studies on observational data when social factors of language use are in question.

We propose a perspective on dialogue that focuses on relative information contributions of conversation partners as a key to successful communication.We predict the success of collaborative task in English and Danish corpora of task-oriented dialogue.Two features are extracted from the frequency domain representations of the lexical entropy series of each interlocutor, power spectrum overlap (PSO) and relative phase (RP).We find that PSO is a negative predictor of task success, while RP is a positive one.An SVM with these features significantly improved on previous task success prediction models.Our findings suggest that the strategic distribution of information density between interlocutors is relevant to task success.

The applicability of entropy rate constancy to dialogue is examined on two spoken dialogue corpora.The principle is found to hold; however, new entropy change patterns within the topic episodes of dialogue are described, which are different from written text.Speaker's dynamic roles as topic initiators and topic responders are associated with decreasing and increasing entropy, respectively, which results in local convergence between these speakers in each topic episode.This implies that the sentence entropy in dialogue is conditioned on different contexts determined by the speaker's roles.Explanations from the perspectives of grounding theory and interactive alignment are discussed, resulting in a novel, unified informationtheoretic approach of dialogue.