We tested how speech phenomena (discourse markers like oh and you know and fillers like uh and um) and the conceptualisation of voices (as human or machine) affected speaker quality ratings (trustworthiness, friendliness, intelligence, and nervousness) across dialogues and monologues. Listeners preferred human voices over synthesised voices in dialogue containing discourse markers and fillers (Study 1). Shifting the location of discourse markers to new locations did not negatively impact ratings of what participants heard (Study 2), but hearing shifted markers did lead participants to rate imagined machine speech containing markers more negatively on a post-experiment questionnaire. Backstories also impacted perceptions of recorded speech: People rated the same synthesised speech more positively if they believed it was created by a person than if they believed it was created by machine (Study 3). Similarly, people's opinions about robots varied depending on context (Study 4). How people rate machine agents in the future will likely reflect how machine agents are portrayed to the public, capabilities aside.

Abstract In a controlled lab experiment, we compared how in-person and robot-mediated communicative settings affected attitudes towards communicators and discourse phenomena related to conversational negotiation. We used a mock interview within-participants experiment design where each participant (mock interviewee) experienced both types of communication with the same experimenter (mock interviewer). Despite communicating with the same person, participants found the in-person interviewer to be more likable, more capable, more intelligent, more polite, more in control, and less awkward than the same person using a telepresence robot. Behaviorally, we did not detect differences in participants’ productions of discourse phenomena (likes, you knows, ums, uhs), laughter, or gaze. We also tested the role of communicative expectations on attitudes towards communications. We primed participants to expect that they would be talking to a person via telepresence, a “disabled” robot-person combination using telepresence, or a person in person (between-participants). We did not find differences arising from people’s expectations of the communication.

Abstract We examined how politeness perception can change when used by a human or voice assistant in different contexts. We conducted two norming studies and two experiments. In the norming studies, we assessed the levels of positive politeness (cooperation) and negative politeness (respecting autonomy) conveyed by a range of politeness strategies across task (Norming Study 1) and social (Norming Study 2) request types. In the experiments, we tested the effect of request type and imposition level on the perception of written requests (Experiment 1) and requests spoken by a voice assistant (Experiment 2). We found that the perception of politeness strategies varied by request type. Positive politeness strategies were rated as very polite with task requests. In contrast, both positive and negative politeness strategies were rated as very polite with social requests. We also found that people expect agents to respect their autonomy more than they expect them to cooperate. Detailed studies of how request context interacts with politeness strategies to affect politeness perception have not previously been reported. Technology designers might find Tables 4 and 5 in this report especially useful for determining what politeness strategies are most appropriate for a given situation as well as what politeness strategies will evoke the desired feeling (autonomy or cooperation).

Despite the pervasiveness of clinical depression in modern society, professional help remains highly stigmatized, inaccessible, and expensive.Accurately diagnosing depression is difficult-requiring time-intensive interviews, assessments, and analysis.Hence, automated methods that can assess linguistic patterns in these interviews could help psychiatric professionals make faster, more informed decisions about diagnosis.We propose JLPC, a method that analyzes interview transcripts to identify depression while jointly categorizing interview prompts into latent categories.This latent categorization allows the model to identify high-level conversational contexts that influence patterns of language in depressed individuals.We show that the proposed model not only outperforms competitive baselines, but that its latent prompt categories provide psycholinguistic insights about depression.

Previous work has shown that conversants adapt to many aspects of their partners' language.Other work has shown that while every person is unique, they often share general patterns of behavior.Theories of personality aim to explain these shared patterns, and studies have shown that many linguistic cues are correlated with personality traits.We propose an adaptation measure for adaptive natural language generation for dialogs that integrates the predictions of both personality theories and adaptation theories, that can be applied as a dialog unfolds, on a turn by turn basis.We show that our measure meets criteria for validity, and that adaptation varies according to corpora and task, speaker, and the set of features used to model it.We also produce fine-grained models according to the dialog segmentation or the speaker, and demonstrate the decaying trend of adaptation.

Personality can be assessed with standardized inventory questions with scaled responses such as “How extraverted is this character?” or with open-ended questions assessing first impressions, such as “What personality does this character convey?” Little is known about how the two methods compare to each other, and even less is known about their use in the personality assessment of virtual agents. We tested what personality virtual agents conveyed through gesture alone when the agents were programmed to display introversion versus extraversion (Experiment 1) and high versus low emotional stability (Experiment 2). In Experiment 1, both measures indicated participants perceived the extraverted agent as extraverted, but the open-question technique highlighted the perception of both agents as highly agreeable whereas the inventory indicated that the extraverted agents were also perceived as more open to new experiences. In Experiment 2, participants perceived agents expressing high versus low emotional stability differently depending on assessment style. With inventory questions, the agents differed on both emotional stability and agreeableness. With the open-ended question, participants perceived the high stability agent as extraverted and the low stability agent as disagreeable. Inventory and open-ended questions provide different information about what personality virtual agents convey and both may be useful in agent development.

Designing virtual characters that are capable of conveying a sense of personality is important for generating realistic experiences, and thus a key goal in computer animation research. Though the influence of gesture and body motion on personality perception has been studied, little is known about which attributes of hand pose and motion convey particular personality traits. Using the “Big Five” model as a framework for evaluating personality traits, this work examines how variations in hand pose and motion impact the perception of a character's personality. As has been done with facial motion, we first study hand motion in isolation as a requirement for running controlled experiments that avoid the combinatorial explosion of multimodal communication (all combinations of facial expressions, arm movements, body movements, and hands) and allow us to understand the communicative content of hands. We determined a set of features likely to reflect personality, based on research in psychology and previous human motion perception work: shape, direction, amplitude, speed, and manipulation. Then we captured realistic hand motion varying these attributes and conducted three perceptual experiments to determine the contribution of these attributes to the character's personalities. Both hand poses and the amplitude of hand motion affected the perception of all five personality traits. Speed impacted all traits except openness. Direction impacted extraversion and openness. Manipulation was perceived as an indicator of introversion, disagreeableness, neuroticism, and less openness to experience. From these results, we generalize guidelines for designing detailed hand motion that can add to the expressiveness and personality of characters. We performed an evaluation study that combined hand motion with gesture and body motion. Even in the presence of body motion, hand motion still significantly impacted the perception of a character's personality and could even be the dominant factor in certain situations.

Amita Misra, Pranav Anand, Jean E. Fox Tree, Marilyn Walker. Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2015.