Recent work has looked to understand user perceptions of speech agent capabilities as dialogue partners (termed partner models), and how this affects user interaction. Yet, partner model effects are currently inferred from language production as no metrics are available to quantify these subjective perceptions more directly. Through three phases of work, we develop and validate the Partner Modelling Questionnaire (PMQ): an 18-item self-report semantic differential scale designed to reliably measure people’s partner models of non-embodied speech interfaces. Through confirmatory factor analysis, we confirm that the PMQ scale consists of three factors: communicative competence and dependability, human-likeness in communication and communicative flexibility. Our studies show that the measure consistently demonstrates good internal reliability, strong test-retest reliability over 4- and 12-week intervals, and predictable convergent/divergent validity. Based on our findings, we discuss the multidimensional nature of partner models, while identifying key future research avenues that the development of the PMQ facilitates. Notably, this includes the need to identify the activation, sensitivity, and dynamism of partner models in speech interface interaction.

A common observation in dialogue research is that people tend to entrain, or align, linguistically with their interlocutors. This phenomenon offers a potentially important way to shape user behavior in human-computer dialogue interactions but little is known about the mechanisms that underlie it and how they may be affected by interlocutor design. We report a Wizard of Oz study that explored how voice anthropomorphism impacts lexical alignment in speech-based human-computer dialogue. In a referential communication task, speakers showed a very strong tendency to align lexical choices with their interlocutors, whether human or computer, but this tendency was not affected by voice anthropomorphism. These results highlight the robustness of lexical alignment effect in speech based human-computer dialogues, and suggest that these effects may be impervious to at least some design cues. They also suggest that automatic priming may be an influential mechanism in explaining why we align lexically with automated dialogue partners.

People display adaptive language behaviors in face-to-face conversations, but will computer users do the same during HCI? We report an experiment (N=20) demonstrating that users' use of language (in terms of lexical choice) is influenced by their beliefs and expectations about a system: When users believe that the system is unsophisticated and restricted in capability, they adapt their language to match the system's language more than when they believe the system is relatively sophisticated and capable. Moreover, this tendency is based entirely on users' expectations about the system; it is unaffected by the actual behavior that the system exhibits. Our results demonstrate that interface design engenders particular beliefs in users about a system's capabilities, and that these beliefs can determine the extent to which users adapt to the system. We argue that such effects can be leveraged to improve the quality and effectiveness of human-computer interactions.