Crying is the language of babies. Inability of parents in understanding the needs of babies causes stress and anxiety. The aim of this study is to develop a rapid prototype of infant-parent communication system. The system was tested on real babies and parents, while they were apart. The Results pointed to the overall acceptance and effectiveness of the system in cry translation and reducing the parents' anxiety with regards to baby cries.

A new approach in telepresence robots via automatic behavior generation is proposed in this paper. The robot behavior is generated using the Smartphone high level sensing, and personality based mood transition. The current telepresence systems require the simultaneous presence of both communication parties. Therefore, due to the time and context differences in distant communication, the opportunity of connectedness is reduced. This is especially important for more intimate telecommunications, that ongoing connectedness is more required. To solve these problems, we developed an automatic behavior generation system, that produces behaviors on behalf of the remote person. In order to be able to infer the state of the remote person more frequently and dynamically, Smartphone sensors are used for automatic high level sensing. Furthermore, to produce more believable affective expressions by the robot, the expressions correspond to the remote user's personality stereotype. Fuzzy Kohonen Clustering Network (FKCN), is utilized to linearly fuse the inferred mood states and regenerate them on the agent. To evaluate the results 120 samples of the user's Smartphone usage and the mood in different times of the day were logged. The state estimated by the model were compared against the users self report data. The results showed that there was no significant difference between the user's self-perceived feelings, and the feelings recognized by the model.

Recent advances in human-robot interaction have enhanced the ability of robots to act as a mediator in remote communication. However, embodied expressive communication across distance is not supported by the current technology. Physical body is a powerful conduit for affective communication, and thus mediating properties of the physical body through an agent has been proposed to address this deficiency. In this paper, we offer heuristic justification on vision of embodied interfaces by adopting perspectives from social psychology in the context of mediated interpersonal communication. Based on our studies, we created a pair of robots named “Mini-Surrogate” as a personalized telecommunication medium which recreates and emulates the physical presence of a specific person. It applies the concepts of enclothed and embodied cognition to convey the illusion of a particular person's presence. Through minimal prototyping in terms of surrogate behaviors, we explored desired nonverbal cues by remotely located couples to satisfy their communication needs while being apart. Our exploratory study pointed to the acceptance of prototyped medium and showed that personalization could enhance communication affectivity in terms of providing awareness and reminding of the specific person. We realized the desired nonverbal cues that helped remote couples to perceive the interface as their remote partner. The findings contribute towards user experience, communication and robotic researchers to enhance the quality of telecommunication for remote couples.

In this paper, we present a new methodology for integrating the sense of taste with the existing digital communication domain. First, we discuss existing problems and limitations for integrating the sense of taste as a digital communication media. Then, to assess this gap, we present a solution with three core modules: the transmitter, form of communication, and receiver. The transmitter is a mobile application, where the sender formulates a taste message to send. Then, for communication, we present a new extensible markup language (XML) format, the TasteXML (TXML) to specify the format of taste messages. As the receiver (actuator), we introduce Digital Taste Stimulator, a novel method for stimulating taste sensations on human. Initial user experiments and qualitative feedbacks were discussed mainly focusing on the Digital Taste Stimulator. We conclude with a brief overview of future aspects of this technology and possibilities on other application domains.

Having dinner together is undoubtedly a kind of pleasurable experience, which involves various channels for mutual interactions, not only audio, vision and touch, but also smell and taste. With the aim to extend this rich experience to remote situations, we developed the Co-dining system to support a range of mealtime interactions to enhance the feeling of social togetherness. This paper describes the preliminary study with this interactive multisensory system. It aims to investigate the actual effectiveness of the working prototype on enhancing the social presence and engagement during telepresent dining, and also to get a comprehensive understanding about users' perception. This evaluation focused on three main aspects: the overall Co-dining feeling, cultural awareness, and engagement. The study results revealed that this system positively achieved the sense of "being together" among users, through the interactive activities touching upon tableware, tablecloth and real edible food, and each interaction module contributed differently to the overall experience. In this paper, we report the evaluation process, present and interpret the data, and discuss the initial insights to enhance the sense of co-presence through multi-channel interactions.

Welcome to the 18th IEEE Virtual Reality conference. As with past conferences, we hope to excite and inspire you with the many events scheduled as part of IEEE VR 2011. Also, we want to emphasize that IEEE VR comes to Asia for only the second time in its long history. It was in Yokohama, Japan in 2001, and this time it will be held in Singapore, one of the most advanced and still rapidly advancing countries in Asia.

Dance is one form of entertainment where physical movement is the key factor. The main reason why robots are experiencing a kind of “boom” is that they have a physical body. We propose a robot dance system that combines these two elements. First, various factors concerning entertainment and dance are studied. Then we propose the dance system by robot using motion unit and the synthetic rule referring the speech synthesis. Also we describe the details of the system by focusing on its software functions. Finally we show the evaluation results of robot dance performances.

This study investigated what effect physical constraints have on the interpretation of demonstrative pronouns when a user navigates a robot. For this investigation, a robot navigation environment called Spondia-II was develope, and an experiment conducted. It is known that the interpretation of demonstrative pronouns requires information about not only the situation (or context) but also the speaker's viewpoint during a dialogue. The results of the experiment suggest that physical constraints do affect the user's viewpoint, especially when a user utters a demonstrative pronoun while navigating the robot. In actual fact, the user alters the use of demonstrative pronouns according to the change in the user's viewpoint. It is also suggested that the user and the robot share the same viewpoint during the physical interaction.

In this paper, we report about a new interaction-oriented robot, which communicates with humans and will participate in human society as our partner. For realizing such a robot, we have started a new collaborative work between cognitive science and robotics. In the way of robotics, we have developed a humanoid robot named "Robovie" that has enough physical expression ability. On the other hand, through cognitive experiments, we obtained important ideas about the robot's body property. To incorporate these ideas, we have developed software architecture and implemented autonomous interactive behaviors to the robot. Further, we have evaluated the robot's performance of the interactive behaviors through psychological experiments. The experiments reveal how humans recognize the robot.

Although the conventional concept of communication is the exchange of logical information, the exchange of Kansei has become very important recently, especially in the case of mobile telephone communications. In addition, it has been recognized that the use of the body, such as facial expressions and gestures, is essential in communications. These facts indicate that technologies that can support Kansei expression using the body will become key technologies for future communications. In this paper, this point is discussed first. Then, as examples of such technologies, research results obtained by the authors are described.

Although the conventional concept of communication is the exchange of logical information, the exchange of Kansei has become very important recently, especially in the case of mobile telephone communications. In addition, it has been recognized that the use of the body, such as facial expressions and gestures, is essential in communications. These facts indicate that echnologies that can support Kansesi expression using the body will become key technologies for future communications. In the paper, this point is discussed first. Then, as examples of such technologies, research results obtained by the authors are described.

In face-to-face communications, the occasional need for intentional lies is something with which everyone can identify. For example, when we get mad, circumstances may force us to put on a big smile instead of expressing our anger; when we feel miserable, good manners may dictate that we greet others warmly. In short, to abide by social norms, we may consciously lie. On the other hand, if we consider the signs that our bodies express as communications (body language), we can say that the body does not lie even when the mind does. Considering this phenomenon, we propose a means of "touching the heart" in a somewhat Japanese way by measuring the heartbeat of the "honest" body and using other technologies to develop a new code of non-verbal communications from a hidden dimension in society. We call this "Meditation art" or "Zen art." "Zen" is Buddhist style meditation.

The creation of interactive animated agents is one of the most active research areas for researchers who are trying to develop various kinds of technologies that give such agents human-like behaviors. However, there are other important issues concerning the agents. One is the introduction of narratives into the behaviors of the agents. Since narratives are the basic way movies and novels take us into cyberspace, they can be effective for creating really human-like agents. Based on this idea, we have developed an interactive movie system (R. Nakatsu and N. Tosa, 1997) in which people can act the main roles and enjoy the story development. At the same time, we have learned that the existence of spontaneous interactions is also the key to the success of this new media. Based on this lesson, we have developed a second system (R. Nakatsu et al., 1999), where we tried to achieve a good balance between narrative-based and spontaneous interactions. In the first part of the paper we introduce our interactive movie system by focusing on how we achieved an integration of narrative-based and spontaneous interactions. Also, there is the important question: what is the objective of creating interactive animated agents? Although the technological aspects of such agents have been studied and discussed in detail, this critical question has rarely been discussed. In the latter part of the paper, we discuss what is the ultimate goal of interactions with interactive agents.

In this paper, we first study the recognition of emotions involved in human speech.We propose an emotion recognition algorithm based on a neural network and also propose a method to collect a large speech database that contains emotions.We carried out emotion recognition experiments based on the neural network trained using this database.An emotion recognition rate of approximately 50% was obtained in a speakerindependent mode for eight emotion states.We then tried to apply this emotion recognition algorithm to a computer agent that plays a character role in the interactive movie system we are developing.We propose to use emotion recognition as key technology for an architecture of the computer characters with both narrative-based and spontaneous interaction capabilities.

In this paper, we first study the recognition of emotions involved in human speech. We propose an emotion recognition algorithm based on a neural network and also propose a method to collect a large speech database that contains emotions. We carried out emotion recognition experiments based on the neural network trained using this database. An emotion recognition rate of approximately 50% was obtained in a speaker-independent mode for eight emotion states.

Nonverbal information, such as the emotions, plays an essential role in human communication. Unfortunately, only the verbal aspect of communication has been focused on in communications between humans and computer agents. In the future, however, it is expected that computer agents will have nonverbal as well as verbal communication capabilities. In this paper, we first study the recognition of the emotions involved in human speech. Then we apply this emotion recognition algorithm to a computer agent that plays a character role in the interactive movie system that we are developing.

Article Interactive movie system with multi-person participation and anytime interaction capabilities Share on Authors: Ryohei Nakatsu ATR Media Integration and Communications Research Laboratories, 2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0288, Japan ATR Media Integration and Communications Research Laboratories, 2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0288, JapanView Profile , Naoko Tosa ATR Media Integration and Communications Research Laboratories, 2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0288, Japan ATR Media Integration and Communications Research Laboratories, 2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0288, JapanView Profile , Takeshi Ochi ATR Media Integration and Communications Research Laboratories, 2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0288, Japan ATR Media Integration and Communications Research Laboratories, 2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0288, JapanView Profile Authors Info & Claims MULTIMEDIA '98: Proceedings of the sixth ACM international conference on MultimediaSeptember 1998 Pages 129–137https://doi.org/10.1145/290747.290764Online:01 September 1998Publication History 3citation389DownloadsMetricsTotal Citations3Total Downloads389Last 12 Months12Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

We propose a new type of speech-based ~nteraction system called "Interactive Poem." Gmventiomd speech-based interaction systems have only focused on the transmission of logical meaning involved in speech.The application of such systems has been restricted to business services such as making reservations or data retrieval.In the Interactive Poem system, however, a human and a computer agent create a poetic worId by exchanging poetic phrases, thus realizing Kansei-based communications between computers and humans.This paper first proposes the concept of "Interactive Poem".Mthen describes the details of the system we have developed, including the software and the hardware configurations as well as the interaction mechanism.1.1 Ke-ywords

We report preliminary results on emotion recognition by machine from joint audiovisual input of facial video and speech. The results show potential advantages in using both modalities over either modality alone. The recognition rate for audio alone is about 75% and video alone about 70%. Using audiovisual data we achieved 97% without increasing the number of features. The improvement in performance is accredited to the complementary property between the two modalities. A possible application is in natural human-computer interfaces.

Real-time spontaneous interaction is the basis of our active behavior such as communication. On the other hand, narratives in movies or novels give us the opportunity to experience dramatic events which are not encountered in our daily lives. The integration of these factors is expected to provide us with a new type of experience. In this paper, by integrating movies and interaction technologies, we propose the concept of "interactive movies". We first explain the concept of interactive movies and describe a prototype system we have developed. We then describe the construction of a second system, which we are currently developing, as well as several improvements in the system.

It is expected that various new types of telecommunications services will emerge based on multimedia technologies. A concept of hypercommunication is proposed that merges human communication in cyberspace involving various people in different places, different times, and even different cultures, as well as communication with human-like agents generated by computers. Technologies necessary to realize this new telecommunications concept are described along with several research examples being done at ATR Media Integration and Communications Research Laboratories.

In the areas of image/speech processing, researchers have long dreamed of producing computer agents that can communicate with people in a human-like way. Although the non-verbal aspects of communications, such as emotions-based communications, play very important roles in our daily lives, most research so far has concentrated on the verbal aspects of communications and has neglected the non-verbal aspects. To achieve human-like agents we have adopted a two-way approach. 1. To provide agents with non-verbal communications capability, engineers have started research on emotions recognition and facial expressions recognition. 2. Artists have begun to design and generate the reactions and behaviors of agents, to fill the gap between real human behaviors and those of computer agents.

A new type of media called 'interactive movies' is proposed which will be realized by the integration of conventional medium such as movies, telecommunications, and video games. The interactive movies have the capability of creating a virtual world with various kinds of realistic/hyper-realistic scenes and computer characters. The metaphor of 'Inter Communication Theater' gives people the ability to interact within this virtual world. Therefore people can experience the stories of the virtual world through interactions with the characters and the environments of the world. In this paper, the basic concept of such a new media will be described along with an experimental system we have developed.

This paper considers the problem of extracting and tracking complex contours without user interaction. We assume that a complex contour consists of contour segments whose spatial coordinates and intensity gradient vary smoothly in the direction normal to themselves. In our algorithm, digital curves that could correspond to contour segments are extracted by connecting edge pixels using a B-spline based contour segment model. The extracted curves trace the contour segments at the next frame by using the active contour model technique. Experimental results show even occluded contours can be tracked automatically.

We describe a method to detect hand position, posture and finger bendings using multiple camera images. Stable detection can be achieved by using skeleton images, and this is confirmed through experiments. This system can be used as a user interface device in a virtual environment, replacing glove-type devices and overcoming most of the disadvantages of contact-type devices. Future work includes verification of the method's availability as a gesture-based man-machine interface system.

This paper describes a speaker-independent isolated word recognition algorithm for telephone voice and its recognition performance. The recognition algorithm consists of two processes ; dynamic time warping and statistical word discrimination. In the first process, input speech is compared with each word template using the dynamic time warping technique. Multiple word templates are used to deal with speech variations among speakers, where each word template is represented by a sequence of phoneme-like templates. To attain high recognition ability, a new technique for generating word templates is proposed. In the second process, statistical word discrimination is carried out for word candidates which have relatively low reliability in the first process. Discrimination functions are calculated based on statistics of transition tendencies of speech characteristics between adjacent frames, and the final word decision is made. The system was trained using utterances from 1305 speakers and tested with utterances from 259 speakers. The average recognition rate of 96.5% was obtained for a 16-word Japanese vocabulary set.

A Japanese text input system using continuous speech recognition is described. This system is composed of two major parts, acoustic processing and linguistic processing. Consonant-Vowel (CV) lattice is generated by acoustic processing from input speech uttered phrase by phrase. CV syllables in continuous speech are detected by continuous dynamic programming (DP) based on SPLIT (Strings of Phoneme-LIke Templates) method. Multiple CV templates extracted from training speech data are used to improve detection accuracy. In linguistic processing, CV lattice is converted into written form using a word dictionary. CV recognition accuracy of 67 % and Japanese translation rate of 53 % are obtained as an experimental result.

Construction and performance of a machine for recognizing spoken connected words are described. In this machine, a new method for connected word recognition, namely inverse dynamic programming (DP) matching, is adopted. Two kinds of DP matching techniques are used in the inverse DP matching, one of which is the usual DP matching and the other is matching performed in a time reverse mode, starting from the end of speech. Combining the similarities obtained by these two kinds of matching, the similarities between input speech and word sequences are computed. Also a technique for rejecting candidates is used in the machine to reduce computation amount. The machine performance is tested by 1400 samples of connected digits. The recognition rate of 99.3% is obtained.

The speech recognition system composing a part of the question-answering system operated by conversational speech is described. The recognition system consists of two stages of process : acoustic processing stage and linguistic processing stage. In the acoustic processing stage, input speech is analyzed and transformed into the phoneme sequence which usually contains ambiguities and errors caused in the segmentation and phoneme recognition. In the linguistic processing stage, the phoneme sequence containing ambiguities and errors is converted into the correct word sequence by the use of the linguistic knowledge such as phoneme rewriting rules, lexicon, syntax, semantics and pragmatics. The voice-operated question-answering system for seat reservation is constructed by computer simulation technique and the promising results are obtained.