Producing realistic character animations is one of the essential tasks in human-AI interactions. Considered as a sequence of poses of a humanoid, the task can be considered as a sequence generation problem with spatiotemporal smoothness and realism constraints. Additionally, we wish to control the behavior of AI agents by giving them what to do and, more specifically, how to do it. We proposed a spatiotemporally-conditioned GAN that generates a sequence that is similar to a given sequence in terms of semantics and spatiotemporal dynamics. Using LSTM-based generator and graph ConvNet discriminator, this system is trained end-to-end on a large gathered dataset of gestures, expressions, and actions. Experiments showed that compared to traditional conditional GAN, our method creates plausible, realistic, and semantically relevant humanoid animation sequences that match user expectations.

According to current estimates, the population of Japan will continuously decrease to be less than 100 million from currently 126.4 million by 2053 [1]. In order to supplement the decreasing labor force, the Japanese government has stated to relax the regulations on introducing foreignworkers. Thus, an increase of foreignerworkers in Japanese society can be expected in the near future. In such a situation, foreigner workers are the minority and have to collaboratively work with Japanese colleagues who are the majority. Our project is aiming at developing an environment for both foreigner and Japanese people to practice collaborative work in an unbalanced situation, that is, Japanese people are the majority and the task itself is Japanese favored. This paper describes our design of an experiment to acquire multimodal sensory data in collaborative tasks with unbalanced composition of cultural backgrounds. These data are supposed to be used for developing foreigner / Japanese behavior generation / detection models in a training environment with virtual agents. From our knowledge, there is no such dataset available and we believe the data collected can provide valuable resources for developing tools in supporting unbalanced groups.

Like a human listener, a listener agent reacts to its communicational partners' non-verbal behaviors such as head nods, facial expressions, and voice tone. When adopting these modalities as inputs and develop the generative model of reactive and spontaneous behaviors using machine learning techniques, the issues of multimodal fusion emerge. That is, the effectiveness of different modalities, frame-wise interaction of multiple modalities, and temporal feature extraction of individual modalities. This paper describes our investigation on these issues of the task in generating of virtual listeners' reactive and spontaneous idling behaviors. The work is based on the comparison of corresponding recurrent neural network (RNN) configurations in the performance of generating listener's (the agent) head movements, gaze directions, facial expressions, and postures from the speaker's head movements, gaze directions, facial expressions, and voice tone. A data corpus recorded in a subject experiment of active listening is used as the ground truth. The results showed that video information is more effective than audio information, and frame-wise interaction of modalities is more effective than temporal characteristics of individual modalities.

We aimed to investigate whether the relationship between the behavior of humans and that of the surrounding agents improves the virtual experience. We conducted an experiment using crowd agents that implement the behavior propagation model based on the herd behavior model. The result indicated that the experiences in the virtual world improved to some extent.

The training program for high school teachers in Japan has less opportunity to practice teaching skills. As a new practice platform, we are running a project to develop a simulation platform of school environment with computer graphics animated virtual students for students’ teachers. In order to interact with virtual students and teachers, it is necessary to estimate the intention of the teacher’s behavior and utterance. However, it is difficult to detection the teacher’s intention at the classroom only by verbal information, such as whether to ask for a response or seek a response. In this paper, we propose an automatic detection model of teacher’s intention using multimodal features including linguistic, prosodic, and gestural features. For the linguistic features, we consider the models with and without lecture contents specific information. As a result, it became clear that estimating the intention of the teacher is better when using prosodic / non-verbal information together than using only verbal information. Also, the models with contents specific information perform better.

An interactive decision-making is useful to put our ambiguous desires into concrete through the interaction with others. However, in human-agent interaction, the agents are often not regarded as well-experienced consultants but rather as human-centered interfaces that provide information. We aimed to induce an active human attitude toward decision-making interactions with agents by controlling the speech reaction time (SRT) of the agents in order to consider the agents as reliable consultants. We conducted an experiment to investigate whether the SRT could influence the human participant's attitude. We used two kinds of agents; one had no SRT (no-SRT) and the other had a SRT of two seconds (2s-SRT). As a result, we found that the no-SRT agents could keep the participants' speech reaction times short even during the decision-making task in which the participants need time for careful consideration. In addition, from the analysis of the number of proposed categories and participant's behavior, we suggest that the participants had an active attitude toward interaction with no-SRT agents.

In the training programs for student teachers, the opportunity to practice teaching skills is often limited due to the lack of resources in preparing a rehearsal environment. We are developing a virtual rehearsal environment for teaching practicing with multiple virtual students. In order to provide feedbacks to the student teachers and allow them to improve their skills, automatic assessment on their performance is required. However, it is hard to assess on teaching because the assessment is subjective and often depends on the tacit knowledge of experienced teacher trainers. In this work, we proposed an automatic assessment model based on human assessment done by experienced high school teachers.

This paper describes our ongoing project to build a platform that enables real-time multimodal interaction with embodied conversational agents. All of the components are in modular design and can be switched to other models easily. A prototype listener agent has been developed upon the platform. Its spontaneous reactive behaviors are trained from a multimodal data corpus collected in a human-human conversation experiment. Two Gated Recurrent Unit (GRU) based models are switched when the agent is speaking or is not speaking. These models generate the agent's facial expressions, head movements, and postures from the corresponding behaviors of the human user in real-time. Benefits from the flexible design, the utterance generation part can be an autonomous dialogue manager with hand crafted rules, an on-line chatbot engine, or a human operator.

Our intellectual life draws on daily conversations that allow us to communicate thoughts, ideas, emotions, etc. To conduct smooth and reliable interactions, participants need to share a solid basis of common ground prior to conversation, which consists of knowledge, beliefs, and suppositions regarding the topics to discuss. Importance of common ground applies to artificial agents as well. A capability of jointly building and maintaining the common ground with people on the fly is indispensable to establish a productive relationship. Understanding and augmenting common ground is challenging, as it is both tacit and dynamic in the sense that the common ground for a situation contains plenty of tacit dimensions and it is dynamically updated as the interaction proceeds.

Our aim is to develop a VR virtual classroom environment that can interact with virtual students through atmosphere. The trainee im- proves his / her class while confirming the change of atmosphere of the virtual students. For such a virtual environment that can give feedback autonomously expressing the atmosphere, data on the atmosphere that can be perceived as feedback are required. In this research, we conducted experiments to verify whether the at- mosphere exuded by virtual students, multi agents, is perceivable through a VR environment.

This paper describes the recent extensions of our previously proposed GECA Framework in incorporating the interconnection with GECA original networking library and ZeroMQ message passing library, the connection with Python based DNN developing API, Keras, and a character animator supporting FACS based detailed facial animations and lip-syncing with real-time generated voice tracks.

Share on Controlling the Timing of Metacognitive Suggestions using Impasse Estimation Authors: Yoshimasa Ohmoto Kyoto University, Kyoto, Japan Kyoto University, Kyoto, JapanView Profile , Hanako Sonobe Kyoto University, Kyoto, Japan Kyoto University, Kyoto, JapanView Profile , Toyoaki Nishida Kyoto University, Kyoto, Japan Kyoto University, Kyoto, JapanView Profile Authors Info & Claims HAI '18: Proceedings of the 6th International Conference on Human-Agent InteractionDecember 2018 Pages 353–355https://doi.org/10.1145/3284432.3287184Published:04 December 2018Publication History 0citation33DownloadsMetricsTotal Citations0Total Downloads33Last 12 Months4Last 6 weeks3 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

This paper presents our proposal of platform with virtual classroom and virtual students which allows to build a flexible system such as autonomous system and evaluation system with combination of modules.

In human-computer interaction, it is first necessary for the human to recognize that there is a meaning beyond information provision while interacting with an agent. One of the critical barriers to social human-computer interaction is the mental state of people with respect to agents. Intentional stance is defined to be an appropriate mental stance in human-agent interaction. To induce intentional stance, we showed interactions between multiple agents in this study. The aim of this study is to investigate whether observing interactions between multiple agents can encourage and maintain the intentional stance toward the agents. We conducted experiments in which participants performed a series of tasks to build a house in a virtual world. In this task, two agents helped and interacted with the participants. The participants were divided into two groups; a group that observed the interaction(observation-group) and a group that directly interacted with the agents(direct-group). From the data obtained in the experiment, we investigated whether the participants tried to interact with the agent that analyzed the participant's behavior and questions. As a result, we could confirm that observing the interaction induced the intentional stance, but the effect was weakened in some cases.

Embodied Conversational Agents often employ advanced multimodal analysis of human users for affective inference, however, its facial expressions in response are often pre-made or coded animations. Generated data-driven facial animations have the advantage that they can be more natural and do not require a database at run-time. The open source modular framework FACSvatar is presented, which processes and animates FACS based data in real-time. Tools that create 3D human models are supported and facial data can be visualized in popular tools in the gaming industry. All functionality is split-up into modules set-up in a publisher-subscriber pattern to provide easy integration with other platforms. A deep learning module for real-time generation of AU data for data-driven animation is implemented. A user evaluation of their expressions being animated through our framework was done. Their ratings were slightly positive, but more improvements have to be made in terms of data quality and individual fine-tuning. Also, the modules' latency and performance have been measured. On average, FACS data is visualized in 28.55 ms.

To establish social relationships between a human and an artificial agent, the agent has to induce and maintain the intentional stance on its human partner. In this study, we focus on contingency, which is the behavior that occurs synchronously with the last action, and the icebreaker, which is a facilitation exercise that helps start an interaction. The aim of this study is to investigate whether an agent that implements contingent responses is capable of inducing and maintaining an intentional stance. We conducted an experiment using the contingent agent and a ``subgoal-oriented agent' as a control group. As a result, we conclude that the contingent responses are capable of maintaining the intentional stance during the main task from the behavior analysis. On the other hand, we suggested that only the participants who actively joined the icebreaker with the contingent agent be induced into taking an intentional stance. From these results, we conclude that the contingent responses could maintain the intentional stance but not induce it.

Virtual agents are useful for simulations of real-world events or processes designed for solving a problem, such as serious games and gamifications. In several types of the virtual world simulations, multiple-to-multiple interaction is needed because our social interaction is multiple-to-multiple in many cases. In this study, we extended the agent behavior model that induces intentional stance in one-to-one human-agent interaction, which was proposed in our previous works, to the multiple-to-multiple human-agent interaction. By using the agent group, we investigated effects of the perspectives that influenced the human mental stance in the multiple-to-multiple human-agent interaction. To investigate the effects of the perspectives, we changed the viewpoints (subjective or objective) in different perspectives (physical or mental) when people interacted with a team of agents in the virtual futsal game. As a result, we can find that there is a different effect between the physical perspective and the mental perspective. The subjective viewpoint of the physical perspective enhanced the feeling of the agent team’s intelligence, but the objective viewpoint of the mental perspective enhanced the feeling of the agent team’s intelligence.

The virtual world simulation has many advantages. However, the effects of the experience strongly influence the mental stance of the participants. To establish social relationships between a human and an artificial agent, the agent has to induce and maintain the intentional stance in its human partner. The purpose of this study was to investigate the method to induce the intentional stance in human participants of one-to-one human-agent interaction by using the alternating estimation of local objectives and global purposes by a network-connected two-layer model. We conducted an experiment to evaluate effect of the proposed method. The results suggested that this method was successful in inducing and maintaining the intentional stance over time.

This paper gives a summary of the 2nd International Workshop on Advancements in Social Signal Processing for Multimodal Interaction (ASSP4MI). Following our successful 1st International Workshop on Advancements in Social Signal Processing for Multimodal Interaction, held during ICMI-2015, we proposed the 2nd ASSP4MI workshop during ICMI-2016. The topics addressed and discussions fostered during last year's workshop are considered very relevant and alive in the research community. In this year's workshop, we continued addressing important topics and fostering fruitful discussions among researchers from different disciplines working in the fields of Social Signal Processing (SSP) and multimodal interaction.

The story base interpretation is subjectively summarised and segmented from the first-person viewpoint. However, we often need to objectively represent an entire image by integrated knowledge. Yet, this is a difficult task. We proposed a novel approach, named the synthetic evidential study (SES), for understanding and augmenting collective thought processes through substantiated thought by interactive media. In this study, we investigated the kind of data that can be obtained through the SES sessions as interpretation archives and whether the database is useful to understand multiple story timelines. For the purpose, we designed a machine-readable interpretation data format and developed support systems to create and provide data that are easy to understand. We conducted an experiment using the simulation of the projection phase in SES sessions. From the results, we suggested that a “meta comment” which was deepened interpretation comment by the others in the interpretation archives to have been posted when it was necessary to consider other participants’ interpretation to broaden their horizons before posting the comment. In addition, the construction of networks to represent the relationships between the interpretation comments enabled us to suggest the important comments by using the degree centrality.

Query-URL relevance, measuring the relevance of each retrieved URL with respect to a given query, is one of the fundamental criteria to evaluate the performance of commercial search engines. The traditional way to collect reliable and accurate query-URL relevance requires multiple annotators to provide their individual judgments based on their subjective expertise (e.g., understanding of user intents). In this case, the annotators’ subjectivity reflected in each annotator individual judgment (AIJ) inevitably affects the quality of the ground truth relevance (GTR). But to the best of our knowledge, the potential impact of AIJs on estimating GTRs has not been studied and exploited quantitatively by existing work. This article first studies how multiple AIJs and GTRs are correlated. Our empirical studies find that the multiple AIJs possibly provide more cues to improve the accuracy of estimating GTRs. Inspired by this finding, we then propose a novel approach to integrating the multiple AIJs with the features characterizing query-URL pairs for estimating GTRs more accurately. Furthermore, we conduct experiments in a commercial search engine—Baidu.com—and report significant gains in terms of the normalized discounted cumulative gains.

In this paper, I present a data-intensive approach to conversational informatics. It not only brings about quantitative understanding, permitting us to turn a great accumulation of keen observations into a pile of computational models, but also helps to build conversational agents by virtue of recent progress in machine learning and data mining. The topics include a smart conversation space for allowing people to engage in conversation in a cyber-physical space, conversational interaction capture for conversation modeling and content production, learning by imitation for producing conversational interactions, cognitive design for understanding mental processes of conversational actors, and synthetic evidential study for collaborative study on understanding social processes.

One barrier to creating truly intelligent autonomous virtual characters is the lack of common ground knowledge contained between the agent and its human interaction partner. Predefining this knowledge for the agent is infeasible so such agents generally can only interact within a limited task domain. This is particularly true for conversational agents, who must handle a wide range of topics. Rather than attempting to tackle the difficult problem of conversation, we instead create an environment where the common ground knowledge is the actions to perform in a virtual basketball game. These actions are not internally predefined, but instead extracted and imitated during interaction. Imitation is crucial because common ground can only be said to exist if there is evidence that the parties have this knowledge [Clark 1996].

Our goal is to make a system to detect the times at which one almost laughed but he or she did not show their laughter on his/her face. We define this kind of laughter as hidden laughter. To accomplish this goal, we first tried making decision trees to detect one's amusement, the input data of which were physiological indices. We used 10-fold cross validation to evaluate the trees, and their accuracy was more than 70%. In addition, we investigated the effect of cultural background on the accuracy.

A promising technology for teaching robots new skills is learning from demonstrations (LfD) or imitation learning. Even though there is currently large literature in imitation learning, smaller attention was given to studying social aspects of the imitation situation. Back imitation is the situation when the demonstrator (human) imitates the learner (robot) before engaging in the actual teaching. In this paper we study the effect of back-imitation on human's perception of the robot's imitative skill. This study is motivated by the need to improve the acceptance of the robot through increasing its perceived imitation skill. Our claim is that back-imitation can be used to achieve this goal. The paper provides two studies involving 24 subjects and 36 HRI sessions that were conducted to investigate this claim. Statistical analysis of the results of these studies show that participants assigned the robots with which they engaged in back-imitation higher scores in terms of imitative skill and overall performance compared with robots that they did not imitate. The paper discusses the implications of this result for the design of imitation situations and broader implications for HRI research.

Query-URL relevance (QUR) is an important criterion to measure the quality of commercial search engines. However, the traditional way to collect high-quality QURs is time-consuming and labor-intensive since it is primarily based on human judges. To address these issues, numerous models have been studied to automatically infer the QURs. Unlike the prior studies in this literature, we first empirically analyze the correlation between multiple annotators' judgments on QURs and URL position in ranking lists. By doing so, we reveal and justify the potential impacts of URL position on inferring intrinsic QURs. Inspired by this finding, a position-sensitive model (PSM) is proposed to infer QURs more accurately. In contrast with most existing approaches that attempt to construct the direct relationship between QURs and the features characterizing query-URL pairs, PSM assumes that the QUR is connected with the features through URL position. We conducted the experiments in real search engine Baidu.com, and compared the experimental results to those of the typical methods used in similar tasks, reporting significant gains over click-through rate and the normalized discounted cumulative gains (NDCGs).

In face-to-face conversations, speakers are continuously checking whether the listener is engaged in the conversation, and they change their conversational strategy if the listener is not fully engaged. With the goal of building a conversational agent that can adaptively control conversations, in this study we analyze listener gaze behaviors and develop a method for estimating whether a listener is engaged in the conversation on the basis of these behaviors. First, we conduct a Wizard-of-Oz study to collect information on a user's gaze behaviors. We then investigate how conversational disengagement, as annotated by human judges, correlates with gaze transition, mutual gaze (eye contact) occurrence, gaze duration, and eye movement distance. On the basis of the results of these analyses, we identify useful information for estimating a user's disengagement and establish an engagement estimation method using a decision tree technique. The results of these analyses show that a model using the features of gaze transition, mutual gaze occurrence, gaze duration, and eye movement distance provides the best performance and can estimate the user's conversational engagement accurately. The estimation model is then implemented as a real-time disengagement judgment mechanism and incorporated into a multimodal dialog manager in an animated conversational agent. This agent is designed to estimate the user's conversational engagement and generate probing questions when the user is distracted from the conversation. Finally, we evaluate the engagement-sensitive agent and find that asking probing questions at the proper times has the expected effects on the user's verbal/nonverbal behaviors during communication with the agent. We also find that our agent system improves the user's impression of the agent in terms of its engagement awareness, behavior appropriateness, conversation smoothness, favorability, and intelligence.

In this paper, we present a novel incremental clustering algorithm which assigns of a set of observations into clusters and learns the distance metric iteratively in an incremental manner. The proposed algorithm SOINN-AML is composed based on the Self-organizing Incremental Neural Network (Shen et al 2006), which represents the distribution of unlabeled data and reports a reasonable number of clusters. SOINN adopts a competitive Hebbian rule for each input signal, and distance between nodes is measured using the Euclidean distance. Such algorithms rely on the distance metric for the input data patterns. Distance Metric Learning (DML) learns a distance metric for the high dimensional input space of data that preserves the distance relation among the training data. DML is not performed for input space of data in SOINN based approaches. SOINN-AML learns input space of data by using the Adaptive Distance Metric Learning (AML) algorithm which is one of the DML algorithms. It improves the incremental clustering performance of the SOINN algorithm by optimizing the distance metric in the case that input data space is high dimensional. In experimental results, we evaluate the performance by using two artificial datasets, seven real datasets from the UCI dataset and three real image datasets. We have found that the proposed algorithm outperforms conventional algorithms including SOINN (Shen et al 2006) and Enhanced SOINN (Shen et al 2007). The improvement of clustering accuracy (NMI) is between 0.03 and 0.13 compared to state of the art SOINN based approaches.

Abstract — Research in robot navigation usually concentrates on implementing navigation algorithms that allow the robot to navigate without human aid. In many real world situations, it is desirable that the robot is able to understand natural gestures from its user or partner and use this understanding to guide its navigation. Some algorithms already exist for learning natural gestures and/or their associated actions but most of these systems does not allow the robot to automatically generate the associated controller that allows it to actually navigate in the real environment. Furthermore, a technique is needed to combine the gestures/actions learned from interacting with multiple users or partners. This paper resolves these two issues and provides a complete system that allows the robot to learn interaction protocols and act upon them using only unsupervised learning techniques and enables it to combine the protocols learned from multiple users/partners. The proposed approach is general and can be applied to other interactive tasks as well. This paper also provides a real world experiment involving 18 subjects and 72 sessions that supports the ability of the proposed system to learn the needed gestures and to improve its knowledge of different gestures and their associations to actions over time. I.

This paper describes an incremental unsupervised clustering mechanism for sequence patterns arising from human gestures. Although self-organizing incremental neural network (SOINN) is known as a powerful tool for incremental unsupervised clustering, it is only applicable to static and fixed-length patterns. In this paper, we propose an extension to SOINN to handle dynamic sequence patterns of variable length. We use a Hidden Markov Model (HMM), as a pre-processor for SOINN, to map the variable-length patterns into fixed-length patterns. HMM contributes to robust feature extraction from sequence patterns, enabling similar statistical features to be extracted from sequence patterns of the same category. As a result of experiments with incremental clustering gesture data, we have found that HMM based SOINN (HB-SOINN) outperforms other methods.

Human-robot interaction using free hand gestures is gaining more importance as more untrained humans are operating robots in home and office environments. The robot needs to solve three problems to be operated by free hand gestures: gesture (command) detection, action generation (related to the domain of the task) and association between gestures and actions. In this paper we propose a novel technique that allows the robot to solve these three problems together learning the action space, the command space, and their relations by just watching another robot operated by a human operator. The main technical contribution of this paper is the introduction of a novel algorithm that allows the robot to segment and discover patterns in its perceived signals without any prior knowledge of the number of different patterns, their occurrences or lengths. The second contribution is using a Ganger-causality based test to limit the search space for the delay between actions and commands utilizing their relations and taking into account the autonomy level of the robot. The paper also presents a feasibility study in which the learning robot was able to predict actor's behavior with 95.2% accuracy after monitoring a single interaction between a novice operator and a WOZ operated robot representing the actor.

The workshop is centered around three main research challenges: 1.) Computationally viable models of cultural aspects of conversations: Cultural norms and values penetrate all our communications and interactions by giving us heuristics how to behave and how to interpret the verbal and nonverbal behavior of others. To make such a notion like culture available for computation, we need a very specific theory of culture that takes its effects on communication and interaction into account.2.) Reliable empirical data on cultural/cross-cultural interaction: To realize technical systems that take cultural influences on behavior into account, precise data analysis on how this influence manifests itself is necessary. In the literature, this information is often given in very general forms without to the precise data on which the observations are based.3.) Enculturating conversational interfaces: Having identified cultural influences on verbal/nonverbal communicative behaviors, it remains to be shown how this can be applied to the development of human-computer interfaces, for instance in an interface reflecting cultural norms and values of communication.

Mutual adaptation is an important phenomenon in human-human communications. Traditionally HRI research was more interested in investigating adaptation of the robot to the human using machine learning techniques but the possibility of utilizing the natural ability of humans to adapt to other humans and artifacts including robots is recently becoming increasingly attractive. This paper presents some of the results from an experiment conducted to investigate the interaction patterns and effectiveness of motion cues as a feedback modality between a human operator and a miniature robot in a confined collaborative navigation task. The results presented in this paper show evidence of human adaptation to the robot and moreover suggest that the adaptation rate is not constant or continuous in time but is discontinuous and nonlinear. The results also show evidence of a starting exploration stage before the adaptation with duration dependent on the expectations of the human regarding the capabilities of the robot in the given task. The paper investigates how to utilize these and related findings for building robots not only capable of adapting to human operators but can also help those operators adapt to them.

Drawing is a very natural activity of humans, and, despite the wide variety of drawing systems available on computers today, most of those systems lack the naturalness of the pencil-paper system. In this paper we present a new drawing interface that is easy for the human to use in a more natural way than the existing drawing interfaces. The proposed system is based on the Interactive Perception paradigm we developed for interfacing social robots to the humans.

Conversation is the most natural communication means for people to communicate with each other. I believe that conversation plays a critical role in realizing a paradigm of human-centered web intelligence in which web intelligence engines are grounded on the human society. We are currently building a computational framework for circulating information in a conversational fashion, using information packages called conversation quanta that encapsulate conversational scenes. Technologies are being developed for acquiring conversation quanta on the spot, accumulating them in a visually recognizable form, and reusing them in a situated fashion. Conversational Informatics constitutes the theoretical foundation for measurement, analysis, and modeling of conversation. I will overview recent results in Conversational Informatics that will help achieve our vision. I will also discuss our approach in the context of Social Intelligence Design aimed at the understanding and augmentation of social intelligence for collective problem solving and learning.

We propose a speech-based conversation environment called Voice Cafe for facilitating dynamic knowledge interaction. Dynamic knowledge interaction is a kind of interaction that makes us creative. Conversation is one of important interactions that facilitates dynamic knowledge interaction. In Voice Cafe, people can have a conversation with conversational agents that can talk with people using synthetic speech. Because conversational agents are equipped with automatic speech recognition (ASR) and text-to-speech (TTS) systems, people can talk with agents, and listen to conversations between agents using speech. Design concepts of the conversation and an overview of Voice Cafe are described.

The authors describe a method for supporting the process of accumulating personal everyday memory and sharing it in a community. Two ideas are used to implement the method. The first idea is intimate computing which enables people to externalize and accumulate personal memory using associative representation at any time and any place. The second idea is a talking virtualized egos metaphor that visualizes community knowledge interaction. This helps community members to find out common interests and shared knowledge. Based on these ideas, we developed the system called CoMeMo which helps people to manage personal and community memory. In experimentation, we observed the following: (a) collaborative knowledge building, (b) change of shared information, (c) creation of new human relationships.

This paper overviews an interdisciplinary study aimed at developing an information technology that facilitates co-evolution of human and knowledge networks in a community. This study combines approaches from social psychology, cognitive psychology and artificial intelligence, to explore a new research field aiming to understand and facilitate dynamic knowledge interaction that may bring about mutual understanding and knowledge evolution in a community. One prominent feature of this study is the use of the public opinion channel (POC) as a common conceptual framework for integrating pieces of work in our approach. The POC is characterized as a novel communication medium that continuously collects messages from people in a community and feeds edited messages back to them. Each work in the interdisciplinary study addresses certain aspects of POC. The POC serves as an important milestone towards dynamic knowledge interaction.

Reinforcement learning is very useful for robots with little a priori knowledge in acquiring appropriate behavior. This paper describes a learning system which can learn a state representation and a behavior policy simultaneously while executing the task. We call the system - the situation transition network system. As cognitive learning, it extracts "situations" and maintains them dynamically in the continuous state space on the basis of rewards from the environment. As behavior learning, it leads to a Markov decision model of environment and performs partial planning on the model. This is a kind of reinforcement learning. The results of computer simulations are given.

We describe information sharing in a community using associative representation. The associative representation links heterogeneous information without defining the semantics strictly. We leave the interpretation of the semantics to human tacit knowledge. We have developed a system called CoMeMo-Community which gathers community members' information automatically and helps the members with sharing it. Our system has a two layer structure that consists of the personal information-space called "Personal Knowledge Layer" and the common information-space called "Common knowledge Layer". In the system, agents that are created by users as the users' alter ego at the Personal Knowledge Layer interact with other agents at the Common Knowledge Layer in place of the users. We have evaluated the system with preliminary experiments. The results suggest that the system enables community members to facilitate communication by sharing each members' information.

We discuss the requirements for information sharing in a community. We propose the weak information structures to integrate heterogeneous information such as static information (e.g. local sites information) and dynamic information created in word-of-mouth communication. We have developed a system using the weak information structures for community information sharing called InfoCommon and evaluated it at the ICMAS'96 Mobile Assistant Project.

The long-term goal of the Knowledgeable Community project is to develop a computational framework for facilitating the knowledge process by humans and computers. We model the knowledgeable community as an open system of natural and artificial agents interacting in knowledgeable environments where real or virtual entities are associated with conceptualization. First, we study associative representation based on information representation with variable degrees of formality which permits raw information materials to be incrementally formalized into a well-defined system of knowledge and even incomplete knowledge pieces to be accessed and utilized by humans and computers. Second, we present the aspect theory for describing the world from multiple perspectives and its application to ontology-based intelligent document base system for human knowledge sharing. Third, we describe a computational framework using agent development, ontology-grounding, and ontology-based mediation. We have implemented several complete prototypes which can demonstrate the significance of our idea. Experimental evaluation is in progress.

Engineers have to refer to various technical information to perform their job. Technical information have been accumulated in an electronic form as documents which contain texts, pictures and movies in enterprises. However, it needs expert knowledge to make use of these technical information. In this paper, we describe an ontology-centric framework, called intelligent corporate base (ICoB) which organizes a technical information into documents, assists a comprehension of documents and helps share information. We explain a prototype system for planning on repair work of transformers. Furthermore, we describe development and use of ontology for technical information sharing.

An agent must acquire internal representation appropriate for its task, environment, and sensors. As a learning algorithm, reinforcement learning is often utilized to acquire the relation between sensory input and action. Learning agents in the real world using visual sensors are often confronted with the critical problem of how to build a necessary and sufficient state space for the agent to execute the task. We propose the acquisition of a relation between vision and action using the visual state-action map (VSAM). VSAM is the application of a self-organizing map (SOM). Input image data is mapped on the node of the learned VSAM. Then VSAM outputs the appropriate action for the state. We applied VSAM to a real robot. The experimental result shows that a real robot avoids the wall while moving around the environment.

This paper describes the CoMeMo project which aims to develop systems which articulate, organize and utilize everyday memory in a group or in a community. The basic hypothesis is that connecting information without defining the semantics using the weak information structures is effective to deal with a large amount of diverse information. The CoMeMo architecture consists of: CoMeMo Workbench, CoMeMo Incorporator, CoMeMo Mobile Terminal, CoMeMo Server, CoMeMo Protocol and CoMeMo Agent. We have implemented the CoMeMo Workbench, the CoMeMo Incorporator and a prototype of the CoMeMo Mobile Terminal. We have developed applications based on the CoMeMo architecture: information gathering and reorganization from WWW pages; ontology development; education support; and conference support.

In this paper, we discuss how ontology plays roles in building a distributed and heterogeneous knowledge-base system. First, we discuss relationship between ontology and agents in the [Formula: see text] which is a framework of knowledge sharing and reuse based on a multi-agent architecture. Ontology is a minimum requirement for each agent to join the [Formula: see text]. Second, we explain mediation by ontology to show how ontology is used in the [Formula: see text]. A special agent called mediator analyzes undirected messages and infer candidates of recipient agents by consulting ontology and relationship between ontology and agents. Third, we model ontology as combination of aspects each of which can represent a way of conceptualization. Aspects are combined either as combination aspect which means integration of aspects or category aspect which means choice of aspects. Since ontology by aspect allows heterogeneous and multiple descriptions for phenomenon in the world, it is appropriate for heterogeneous knowledge-base systems. We also show translation of messages as a way of interpreting multiple aspects. A translation agent can translate a message with some aspect to one with another aspect by analyzing dependency of aspects. Mediation and translation of messages are important to build agents easily and naturally because less knowledge on other agents is requested for each agent.

Identifying inaccurate data has long been regarded as a significant and difficult problem in AI. In this paper, we present a new method for identifying inaccurate data on the basis of qualitative correlations among related data. First, we introduce the definitions of related data and qualitative correlations among related data. Then we put forward a new concept called support coefficient function (SCF). SCF can be used to extract, represent, and calculate qualitative correlations among related data within a dataset. We propose an approach to determining dynamic shift intervals of inaccurate data, and an approach to calculating possibility of identifying inaccurate data, respectively. Both of the approaches are based on SCF. Finally we present an algorithm for identifying inaccurate data by using qualitative correlations among related data as confirmatory or disconfirmatory evidence. We have developed a practical system for interpreting infrared spectra by applying the method, and have fully tested the system against several hundred real spectra. The experimental results show that the method is significantly better than the conventional methods used in many similar systems.

Due %0 the advance of electronic instrumentation, auwmsted ship increases in number.)

This paper proposes a method for organizing linguistic knowledge in both systematic and flexible fashion. We introduce a purely applicative language (PAL) as an intermediate representation and an object-oriented computation mechanism for its interpretation. PAL enables the establishment of a principled and well-constrained method of interaction among lexicon-oriented linguistic modules. The object-oriented computation mechanism provides a flexible means of abstracting modules and sharing common knowledge.

This paper describes an analysis method which uses heuristic knowledge to find local syntactic structures of Chinese sentences. We call it a preprocessing, because we use it before we do global syntactic structure analysis (1) of the input sentence. Our purpose is to guide the global analysis through the search space, to avoid unnecessary computation.To realize this, we use a set of special words that appear in commonly used patterns in Chinese. We call them "characteristic words". They enable us to pick out fragments that might figure in the syntactic structure of the sentence. Knowledge concerning the use of characteristic words enables us to rate alternative fragments, according to pattern statistics, fragment length, distance between characteristic words, and so on. The preprocessing system proposes to the global analysis level a most "likely" partial structure. In case this choice is rejected, backtracking looks for a second choice, and so on.For our system, we use 200 characteristic words. Their rules are written by 101 automata. We tested them against 120 sentences taken from a Chinese physics text book. For this limited set, correct partial structures were proposed as first choice for 94% of sentences. Allowing a 2nd choice, the score is 98%, with a 3rd choice, the score is 100%.