We present the Brooklyn Multi-Interaction Corpus (B-MIC), a collection of dyadic conversations designed to identify speaker traits and conversation contexts that cause variations in entrainment behavior. B-MIC pairs each participant with multiple partners for an object placement game and open-ended discussions, as well as with a Wizard of Oz for a baseline of their speech. In addition to fully transcribed recordings, it includes demographic information and four completed psychological questionnaires for each subject and turn annotations for perceived emotion and acoustic outliers. This enables the study of speakers’ entrainment behavior in different contexts and the sources of variation in this behavior. In this paper, we introduce B-MIC and describe our collection, annotation, and preprocessing methodologies. We report a preliminary study demonstrating varied entrainment behavior across different conversation types and discuss the rich potential for future work on the corpus.

Andreas Weise, Vered Silber-Varod, Anat Lerner, Julia Hirschberg, Rivka Levitan. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 2021.

Humans express ideas, beliefs, and statements through language. The manner of expression can carry information indicating the author’s degree of confidence in their statement. Understanding the certainty level of a claim is crucial in areas such as medicine, finance, engineering, and many others where errors can lead to disastrous results. In this work, we apply a joint model that leverages words and part-of-speech tags to improve hedge detection in text and achieve a new top score on the CoNLL-2010 Wikipedia corpus.

Entrainment, the phenomenon of conversational partners’ speech becoming more similar to each other, is generally accepted to be an important aspect of human-human and human-machine communication. However, there is a gap between accepted psycholinguistic models of entrainment and the body of empirical findings, which includes a large number of unexplained negative results. Existing research does not provide insights specific enough to guide the implementation of entraining spoken dialogue systems or the interpretation of entrainment as a measure of quality. A more integrated model of entrainment is proposed, which looks for consistent explanations of entrainment behavior on specific features and how they interact with speaker, session, and utterance characteristics.

Depression is a serious illness that affects millions of people globally. In recent years, the task of automatic depression detection from speech has gained popularity. However, several challenges remain, including which features provide the best discrimination between classes or depression levels. Thus far, most research has focused on extracting features from the speech signal. However, the speech production system is complex and depression has been shown to affect many linguistic properties, including phonetics, semantics, and syntax. Therefore, we argue that researchers should look beyond the acoustic properties of speech by building features that capture syntactic structure and semantic content. We provide a comparative analyses of various features for depression detection. Using the same corpus, we evaluate how a system built on text-based features compares to a speech-based system. We find that a combination of features drawn from both speech and text lead to the best system performance.

Entrainment is the tendency of speakers engaged in conversation to align different aspects of their communicative behavior. In this study we explore in more detail a measure of prosodic entrainment defined in previous work, which uses a discrete parametrization of intonational contours defined by the ToBI conventions for prosodic description. We divide this measure into two asymmetric variants: backward mimicry (in which a speaker uses a contour used previously by the interlocutor) and forward influence (in which a speaker’s contour appears later in the speech of the interlocutor). This distinction sheds new light on significant correlations with a number of social variables related to the level of engagement of speakers in a corpus of task-oriented dialogues in Standard American English.

It is well established that speakers of Standard American English entrain, or become more similar to each other as they speak, in acoustic-prosodic features of their speech as well as other behaviors.Entrainment in other languages is less well understood.This work uses a variety of metrics to measure acoustic-prosodic entrainment in four comparable corpora of task-oriented conversational speech in Slovak, Spanish, English and Chinese.We report the results of these experiments and describe trends and patterns that can be observed from comparing acoustic-prosodic entrainment in these four languages.We find evidence of a variety of forms of entrainment across all the languages studied, with some evidence of individual differences as well within the languages.

Entrainment is the propensity of conversational partners to align different aspects of their communicative behavior. In this study we present three novel measures of prosodic entrainment based on intonational contours as defined by the ToBI conventions for prosodic description. Each of these measures estimates the similarity of contours used by speakers in different ways: by means of the perplexity of n-gram models, the Levenshtein distance, and the Kullback-Leibler divergence measure. We report significant correlations between each of these measures and manual annotations of a number of social variables related to the level of engagement of speakers, in a corpus of task-oriented dialogues in Standard American English.

We describe acoustic/prosodic and lexical correlates of social variables annotated on a large corpus of task-oriented spontaneous speech.We employ Amazon Mechanical Turk to label the corpus with a large number of social behaviors, examining results of three of these here.We find significant differences between male and female speakers for perceptions of attempts to be liked, likeability, speech planning, that also differ depending upon the gender of their conversational partners.

In conversation, speakers become more like each other in various dimensions.This phenomenon, commonly called entrainment, coordination, or alignment, is widely believed to be crucial to the success and naturalness of human interactions.We investigate entrainment in four acoustic and prosodic dimensions.We explore whether speakers coordinate with each other in these dimensions over the conversation as a whole as well as on a turn-by-turn basis and in both relative and absolute terms, and whether this coordination improves over the course of the conversation.