In this study, we developed a communication robot that supports multi-party human conversations and helps participants recover from awkward silences. To designate the next speaker, three types of robot behaviors are considered, i.e., promoting a specific individual to talk, promoting all participants to talk, and extracting utterances from participants using undirected muttering. We developed a conversation robot that can selectively designate the next speaker on the basis of these behaviors in a conversation between three humans and a robot (Figure 1). Evaluation tests and case studies were performed relative to speech obligations, pressure to speak and cooperative responses among participants. In addition, methodologies for designing communication support robots that attempt to encourage conversation among reserved participants are discussed.

When there is a lull in conversation, many people feel awkward and make sounds, such as “ummm” (a vocal filler), or stroke their chins or other parts of their bodies (a bodily filler). These fillers suggest the intent to recommence and continue the conversation. Thus, the purpose of conversation is to facilitate the sharing of beneficial information as well as a comfortable moment with an interlocutor. In this manner, a robot intended to help foster a comfortable or relaxing conversational atmosphere through interaction with a user (e.g., therapeutic robots) needs to be designed to convey such a cooperative demeanor to its human interlocutor. In this study, we analyze the effects of a robot's vocal and bodily fillers during awkward silences between turns in conversations with humans. The results of our study show that people feel awkward during silences, even when in conversation with a robot, and that the robot's conversational fillers help mitigate awkwardness and express a cooperative attitude in verbal interactions with people. Our experiments also revealed that subjects who were less socially adept reported feeling that their robot interlocutor was more sincere than its human counterpart.

In this study we analyzed the roles of verbal/bodily fillers for recovering from awkward silences in conversations. We focused on verbal fillers such as “ummm” and “uh,” and bodily fillers like “touching own hair or chin” that commonly emerge during silences between turns in conversations. We designed and created simulated dyadic-conversation scenarios using computer graphics characters, and then performed evaluations utilizing stimuli drawn from these simulations. Subjective evaluation results suggested that fillers express participants' sincerity in maintaining conversations and they can be used as clues for other participants to begin their utterances. These findings have practical implications for the behavioral design of conversational robots that can behave more appropriately and politely with humans.

The appearance and behavioral design of life-like agents used in human-computer interaction is important to establish an intuitive and emotional relationship between a user and an agent. This paper analyzes the impressions of a “sidelong glance” as one of the combinatorial nonverbal behaviors that comprises gaze, head orientation, and facial expression. Through the analysis, we show that the “sidelong glance” conveys the specific impression of “friendliness” and “dominance” on users. We also show that an “attractive face” is friendlier than the “average face” and that a “smiling face” emphasizes friendliness. Finally, on the basis of our experimental results, a comprehensive affection model of agent behaviors is proposed. The model will contribute to designing the appearance and behaviors of interactive life-like agents.

Human eye gaze is a strong candidate to create a new application area based on human-computer interaction. To implement a really practical gaze-based interaction system, gaze detection must be realized without placing any restriction on the user's behavior or comfort. This paper describes a gaze tracking system that offers freehead, simple personal calibration. It does not require the user wear anything on her head, and she can move her head freely. Personal calibration takes only a very short time; the user is asked to look at two markers on the screen. An experiment shows that the accuracy of the implemented system is about 1.0 degrees (view angle).

This paper proposes a video cut editing rule based on participants' gaze for extracting and conveying the flow of conversation in multiparty conversation. Systems that record meetings and those that support teleconferences are attracting considerable interest. Conventional systems use a fixed-viewpoint camera and simple camera selection based on participants' utterances. However, conventional systems fail to convey a sufficient amount of nonverbal information about the participants and the flow of conversation. We focus on participants' gaze since it is a good indicator of the participants' intent and emotion, conversational attention etc. We propose a video cut editing rule based on the convergence of participants' gaze direction. We conduct an experiment to evaluate the effectiveness of the proposed method. The results indicate that the proposed method can successfully convey who is taking to whom, which is a key indicator of the flow of conversation.

We propose a gaze movement model that enables an embodied interface agent to convey different impressions to users. Managing one's own impression to influence the behaviors of others plays an important role in human communications. To create a new application area which involves agents in this kind of social interaction, interface agents that manage their impressions are required. For this purpose, we build the gaze movement model based on three gaze parameters picked from a large number of psychological studies: amount of gaze, mean duration of gaze, and gaze points while averted. In this paper, we introduce the gaze movement model and gaze parameters. We then present an experiment in which subjects evaluated the impressions created by nine gaze patterns produced by altering the gaze parameters. The results indicate that reproducible relations exist between the gaze parameters and impressions, which shows the validity of the model

In this paper we introduce a novel gaze tracking system called FreeGaze, which is designed for the use of everyday gaze interaction. Among various possible applications of gaze tracking system, Human-Computer Interaction (HCI) is one of the most promising elds. However, existing systems require complicated and burden-some calibration and are not robust to the measurement variations. To solve these problems, we introduce a geometric eyeball model and sophisticated image processing. Unlike existing systems, our system needs only two points for each individual calibration. When the personalization nishes, our system needs no more calibration before each measurement session. Evaluation tests show that the system is accurate and applicable to everyday use for the applications.

Conventional features are robust for recognizing either deformed or degraded characters. This paper proposes a feature extraction method that is robust for both of them. Run-length compensation is introduced for extracting approximate directional run-lengths of strokes from degraded handwritten characters. This technique is applied to the conventional feature vector based on directional run-lengths. Experiments for handwritten characters with additive or subtractive noise show that the proposed feature is superior to conventional ones over a wide range of the degree of noise.

A novel method for multispectral image retrieval is presented. This method uses a representation of image features based on vector quantization. Feature representation is important for image retrieval, but there are difficulties in applying conventional histogram-based representations to multispectral images. We developed an efficient feature representation and a novel method for the retrieval of images by quantizing each image adaptively.

In this paper a novel method for the unsupervised segmentation of textured images is presented. Textures are modeled as spatial interactions between pixels. Thus, a certain window size is required to extract texture features and to estimate texture boundaries by using the features. As long as the size of the window is fixed over the whole of an image, we cannot accurately estimates texture regions that have similar properties. In this paper the problem of selection of an appropriate size for window used to estimate homogeneous texture regions is investigated via hypothesis and testing. Experiments on segmentation of textures in synthetic and natural images show the effectiveness of the method.

A multiple observation-based scheme (MObS) is described for robust facial recognition, and a novel object recognition method called kernel mutual subspace method (KMS) is proposed. The mutual sub-space method (MSM) proposed by (Maeda, et al., 1999) is a powerful method for recognizing facial images. However, its recognition accuracy is degraded when the data distribution has a nonlinear structure. To overcome this shortcoming we apply kernel principal component analysis (kPCP) to MSM. This paper describes theoretical aspects of the proposed method and presents the results of facial image recognition experiments.

This paper proposes a novel object-based method to similar image retrieval by representing an image by its salient object regions as well as associated color texture and shape features. The major obstacle in developing such methods is the difficulty of accurately segmenting the image into prominent regions. To overcome this difficulty we applied an inhomogeneous diffusion model to both color and texture features to quasi-accurately detect salient regions in the image. Characteristic color texture and shape features of these regions are obtained on the basis of diffusion results. These features are invariant to rotation. Scale invariance was also achieved for the color and shape features. A suitable user interface was developed to facilitate object-based searches. Enabling the user to specify the objects he/she is looking for ensures a higher level of performance and flexibility. Experiments conducted on a large number of images taken from Corel and Kodak photo-CD data show that the method performs well for a large variety of natural images.

This paper presents a new method for reconstructing object surface from sparse data while preserving discontinuities. The proposed method is an extension of regularization by applying a smoothness map as an array of regularization parameters. Existing methods that preserve discontinuity simply change the array of regularization parameters based on local features. Therefore, the results using these methods are affected by not only local features but also noise. This paper proposes a method to obtain the smoothness map by taking the global structure of the shape into consideration. To evaluate the global structure, the smoothness map should be described as simply as possible. We apply the minimum description length (MDL) principle to obtain an appropriate smoothness map as it describes the best surface model. Then, we show that the smoothness map can be estimated by minimizing the description length for the smoothness map while fitting the surface to the observed data. As a result, the proposed method can reconstruct object surfaces while preserving discontinuities using the estimated smoothness map. Experiments performed on real scenes are shown.

We propose a method for detecting changes of buildings by comparing aerial images which are taken at different times. In the proposed method, we consider the shadow and shading model. The model is derived from a relation between incident light of the Sun and 3D shapes of buildings which intercept the incident light. The advantage of the proposed method is that we only compare 2D aerial images based on the model without estimating 3D shapes of buildings. When we compare the aerial images, we have the problem that changes of shadows and buildings are observed simultaneously in the images. To solve this problem, we propose criteria for distinguishing the changes of shadows and buildings. Our experiments have shown that the proposed method can distinguish the changes of shadows and buildings in real aerial images.

A new approach to optical character recognition (OCR) is presented. Whereas existing OCR systems have been designed 20 obtain sufficient recognition accuracy, even though this does not provide any direct information useful for improving the systems, this approach uses genetic algorithms to analyze the feature space of the system by visualizing the forms of character images that correspond to the feature vectors in a way that humans can comprehend. It is shown that character images can be reconstructed from feature vectors by 100-generation iteration using genetic algorithms. Experimental results for visualizing reference vectors and category boundaries are presented. The results suggest the existence of ambiguous regions in category boundaries.

This paper describes a method for controlling the trajectory tracking of a mobile object, enabling the mobile object to accomplish some task involving a target object. A sequence of image feature points is used to express the reference trajectory that the mobile object should track and the mobile object is controlled by a feedback control law derived from the sequence of image feature points. This control is robust and flexible because the mobile object tracks a reference trajectory in an object-centered coordinate system of the target object. Moreover, the trajectory can be taught with only an image sequence. Simulation showed that a mobile object can track a trajectory described in an object-centered coordinate system.

The authors provide an object representation method that describes 3-D shape and pose using a set of covariant quantities based on a group theoretical approach. These covariant quantities can produce coordinate-invariant features that represent an object's shape and coordinate variant features that represent its pose. First they formulate covariant quantities that are linearly transformed when the coordinate system is changed. Then they show that the invariant features capture coordinate-independent shape information and provide descriptors for indexing into a database of stored models. The variant features are obtained from the covariant quantities in terms of pose equations. A sufficient number of equations (at least three) can determine the object pose. The authors describe two pose estimation algorithms that use these equations, and present numerical simulation results which demonstrate that the algorithms yield close approximations of the object poses.

A comprehensive scene analysis algorithm that extracts object shape, reflection coefficients and illuminant direction from just sequential images expressed as luminance values is proposed. The proposed algorithm uses an optical model consisting of a light source, an object and a viewer. Within this framework, the algorithm allows for diffuse and specular reflection, variations in object reflection coefficients and shading. The first phase of the algorithm calculates three values: (1) the variation of inner product of illuminant direction and surface normal before and after motion, and (2) the variation of shading, as well as (3) the motion field. The second phase uses these values to recover not only object shape but also illuminant direction and reflection coefficients of the surface. The algorithm's performance is confirmed through computer simulation. The reflection coefficients and the light source direction are estimated accurately while the recovered shape contains some errors.>

A motion picture coding algorithm using motion-compensated interframe prediction and the adaptive discrete cosine transform (DCT) encoding technique is proposed. High coding efficiency is obtained by the adaptive DCT encoding technique in which encoding parameters are fitted to widely varying characteristics of the interframe differential signal. Segmented DCT subblocks of interframe prediction error are classified into categories based on their coefficient power distribution characteristics. The adaptation gain results from using a suitable variable word length code set designated by the above classification for encoding each quantization index of DCT coefficients. In addition, a new coding parameter control method is introduced based on the information rate estimation of the current frame. This classification promotes high stability because good estimation accuracy of bits consumption for each DCT subblock is obtained by utilizing the category indexes. Simulation results show that the proposed algorithm has enough coding efficiency to transmit videoconferencing motion pictures through a 384 kbit/s channel.

This paper describes custom designed VLSI coder and decoder, which transmit time division multiplexed (TDM) color signals at 32 Mbit/sec. Two-dimensional intraframe DPCM prediction, variable word length coding, and data buffer to smooth the data rate are devised on one coder chip. Reverse function are devised on one decoder chip. Test sequence is investigated for logic design purposes. Several techniques, e.g., double phased clock and prediction loop modification are employed to simplify the configuration and to ensure the operation rate of video processing.

Various interframe coding methods have been devised to reduce the cost of television signal transmission. Some degradation still remains, however, because prediction efficiency is lowered in uncovered background areas which appear behind moving objects. This paper proposes an uncovered background predictive coding algorithm, which increases prediction efficiency. Prediction efficiency is evaluated by using a simple signal model and real television signals. Experimental results show that the information generation rate is reduced to 78 percent as compared to that of previous frame predictive coding when the displacement is 10 pels/frame.

This paper describes the interframe coding system (TRIDEC 1.5) used for NTSC color television signal transmission at a 1.5 Mbit/s rate. TRIDEC 1.5 is developed with the intention of making video teleconferencing service more economical. The system utilizes the following highly efficient coding techniques: 1) interframe combinational difference coding 2) temporal subsample coding 3) noise reduction 4) demand refreshing to protect against channel errors, which makes periodic forced refreshing unnecessary. In addition, new coding parameter control methods have been devised to prevent buffer memory overflow. The subjective evaluation test results make it clear that the encoded picture quality is good enough for teleconferencing use.