In our daily life, the interactive roles of leaders, followers, and coordinators tend to emerge from multiparty collaboration. The primary purpose of this study is to automatically predict the leading role in multiparty interaction by ubiquitous computing techniques. Even though the leading role has been predicted for an entire task, there has been little focus on evaluating how roles are reorganized during a task. To find the verbal and nonverbal cues that might predict roles, we asked neutral third parties to select the participant playing the leading role in an assembly task. We examined the correlation between behavioral data gathered during a task and third-party evaluations of the leading role player in terms of temporal alterations. The preliminary results suggest that task-oriented utterances and verification behaviors regarding progress status contribute to the prediction of the emerging and reorganized leader. Moreover, we discuss the implications of our findings for the design of applications that can enhance multiparty collaboration.

Androids that replicate humans in form also need to replicate them in behaviour to achieve a high level of believability or lifelikeness. We explore the minimal social cues that can induce in people the human tendency for social acceptance, or ethopoeia, toward artifacts, including androids. It has been observed that people exhibit a strong tendency to adjust to each other, through a number of speech and language features in human–human conversational interactions, to obtain communication efficiency and emotional engagement. We investigate in this paper the phenomena related to prosodic alignment in human–computer interactions, with particular focus on human–computer alignment of speech characteristics. We found that people exhibit unidirectional and spontaneous short-term alignment of loudness and response latency in their speech in response to computer-generated speech. We believe this phenomenon of prosodic alignment provides one of the key components for building social acceptance of androids.

The aim of this paper is to clarify the relationship between the expressive strengths of gestures and voice for embodied and personified interfaces. We conduct perceptual tests using a puppet interface, while controlling singing-voice expressions, to empirically determine the naturalness and strength of various combinations of gesture and voice. The results show that (1) the strength of cross-modal perception is affected more by gestural expression than by the expressions of a singing voice, and (2) the appropriateness of cross-modal perception is affected by expressive combinations between singing voice and gestures in personified expressions. As a promising solution, we propose balancing a singing voice and gestural expressions by expanding and correcting the width and shape of the curve of expressive strength in the singing voice.

We have developed a method for synthesizing a singing voice by gradually changing the musical expression based on speech morphing. This paper shows the advantages of this method in comparison with the approach of binary discrete transformation between two expressions, confirmed by statistical analyses of perception tests. In order to synthesize different expressional strengths of a singing voice, a "normal" (without expression) voice of a particular singer is used as the base of morphing, and three different expressions, "dark," "whispery" and "wet," are used as the target. Through our experiments, we confirmed i) the proposed morphing algorithm effectively interpolates the expressional strength of a singing voice, ii) an approximate equation of the perceptual sense can be used to calculate the morph ratio at a perceptually linear interval, and iii) our gradual transformation method can generate a natural singing voice from the interpolation of two different expressions.

An adaptive speech recognizer is a key function in the design of a robust spoken dialogue system. Our research focuses on the human tendency of prosodic alignment to one’s conversational partners. A spoken dialogue system might be able to exploit this human tendency to implicitly influence people to manage their speech at the prosodic level in order to accommodate its recognition capabilities. Consequently, this would decrease recognition errors. Prosodic alignment in human-computer interaction has been studied as part of the problems of personality alignment in the context of animated conversational characters. The present study examines human prosodic alignment tendency at more micro level, and explores whether people’s speech amplitude and pause length align to those of computer generated voices within a dialogue exchange. We found that people exhibit spontaneous short-term alignment of speech prosody to the slight prosodic changes in a computer’s voice within a session, even without the help of animated conversational characters.

This paper examines whether humans perceive additional meaning in computer utterances with different voice prosody and change their decisions based on such interaction if the prosodic differences carry informational significance. We conduct a route selection experiment in which participants were asked to find a route to a goal in a 3-D maze generated by computer graphics. The maze system occasionally outputs a confirmation in response to the participant’s choice of a route at a junction. The prosodic characteristics of the confirmation utterances reflect whether the route selected is right for reaching the goal in a normal alley or wrong, leading to a dead end. In this experiment, participants are able to recognize the prosodic meaning and successfully make a decision that leads to a route through the maze. This result suggests that participants can perceive the differences in pitch range of the confirmation voice, and used the pitch range in reselecting their route. Furthermore, wide pitch range confirmation voice indicates a dead end resulted in a higher reselection ratio. This suggests that participants gave a negative interpretation to wider pitch range responses.

This paper describes the development of a portable speech interpreter by which the user can look for words and relating sentences easily using his/her voice. The prototype uses a RISC microprocessor for the embedded use, resulting in a compact body of 15cm(W) and 6cm(H). The voice user interface is helpful, especially in small devices, but it has a serious problem related to the recognition errors. The total user interface must be designed carefully to minimize the inconvenience caused by the recognition errors. We have developed an interface in which the confinement of the dictionary realizes the quick and accurate recognition, and the syllable correction procedure provides an easy way to correct recognition errors. These features make it possible to input any words, which are found in a standard Japanese dictionary. Experimental evaluations show that the N-best recognition rate for words and syllables are acceptable.

The paper presents an evaluation method to assess impressions toward a computer-generated creature through human-computer interaction. The aim of our study is to construct a mechanism on a computer to realize empathic interaction between humans and creatures. In this paper, we examine the e ect on personality cognition of a virtual creature when the creature replies with di erent timing to a human utterances. The creature has an abstract appearance and a non-linguistic voice. We control the following three conditions for timing of the creature's response: a) alternation, b) overlap, and c) neutral condition. As a result, we found that the evaluation value of the alternation condition shows more positive behavior than that of the overlap and neutral conditions when the creature has the above features.

In this paper, we present a general framework and architecture for maintaining dialogue coordination in spoken dialogue systems, in which intended behaviors and goals are incrementally performed during the course of maintaining dialogue coordination. The dialogue structure emerges as a result from interaction between user and the dialogue system. The key feature of this design for the systems is to use multiple situated-agents for coordinating communicative acts that are realized as a hierarchy of autonomous behaviors by using a subsumption architecture. In this architecture it should be noted that the lower-level behaviors act autonomously for maintaining the dialogue coordination and are linked to the specifications from higher-level behaviors for dialogue management. In order to make the behavior of the system social, in general, the maintaining of dialogue coordination takes priority over the realization of intended goals of the system as a dialogue participant. We introduce an under-specification strategy for controlling the preference of the concurrent behaviors. This is in contrast to the classical, top-down approach to dialogue coordination.

This paper discusses the effectiveness of human-computer interaction in our prototype system “Talking Eye”, which is based on social behaviors in self-motivated dialogue. Talking Eye system consists of autonomous creatures having the emergent computation architecture with the advantage of selfmotivated dialogue. We performed an experiment using subject’s impression to illustrate the effectiveness of selfmotivated dialogue with the Talking Eye system. The result was obtained that they could find personal nature during interaction even if they could not accomplish their conversational purpose. The main goal of this work is to build a mechanism for autonomous creatures that promotes a more consensual feeling through its interaction with humans.