With the rapid advancement of robotics, robots’ helping behaviors are increasingly framed not only as functional assistance but also as prosocially meaningful interaction. In this context, the resource cost borne by the help-provider is a critical factor, yet it has not been systematically explored in existing human-robot interaction (HRI) research. Understanding how humans perceive and respond to different types of helping is essential for building better human–robot relationships. This study addresses this gap through two experiments. Study 1 examined the role of agent resource type (robot’s own resources vs. external resources). Results showed that when robots shared their own resources, participants did not report significant differences in overall attitudes or prosocial behavior, but attributed higher performance trust and expressed stronger feelings of gratitude and guilt. Study 2 further examined the importance of agent resources (robot battery level: high vs. low) . The results showed that even when relative costs were the same, participants tended to perceive sharing from a low-battery robot as more reliable, while variations in resource type or importance did not significantly change social responses. These findings suggest that human evaluations of robots are shaped not only by the outcomes of helping but also by the perceived cost and sacrifice underlying robot actions. Our work offers an initial direction for integrating resource cost considerations into the design of social robots.

In this study, we explore how robot self-sacrifice can influence human perceptions and behaviors toward robots. While traditional research in human-robot interaction (HRI) often addresses moral dilemmas, such as the trolley problem, our work examines more relatable scenarios where robots engage in self-sacrificial behavior, such as offering their own battery to charge a user's device instead of relying on external resources. Through an experiment with 30 participants, we found that robots demonstrating self-sacrifice significantly promoted prosocial behaviors compared to robots that did not. However, no significant differences were observed between the groups in terms of the perceptions of robots. These results highlight that while self-sacrificial behavior did not alter perceptions of the robot's social traits, it clearly influenced participants' willingness to engage in prosocial actions. This research underscores the potential of robots to foster prosocial behavior through self-sacrifice, offering valuable insights for designing robots that encourage a flourishing society in which humans and robots coexist.

Trust in AI is crucial for a cooperative relationship between humans and AI in systems utilizing this technology, such as autonomous driving systems. Trust facilitates the appropriate utilization of these systems, thereby optimizing their potential benefits. If humans over-trust or under-trust an AI, serious problems such as misuse and accidents occur. To prevent over/under-trust, it is necessary to estimate trust. However, trust is an internal state of humans and is hard to observe directly. Therefore, we propose a estimation model for trust using dynamic structure equation modeling, which extends SEM and can handle time-series data. A path diagram, which shows causalities between variables, is developed in an exploratory way and the resultant path diagram is optimized for effective path structures. Over/under-trust was estimated with 99 % accuracy in an autonomous driving task. These results show that our proposed method outperformed the conventional method including an auto regression family.

This study investigates the role of robot bowing in trust repair, focusing on how human-like movement impacts trust in human-robot interaction. We manipulated the movement quality across four conditions: human-like, constant-speed, abrupt, and no-movement. Specifically for the human-like movement, we analyzed the Japanese hospitality (called “omotenashi”) gesture, which expresses attentiveness and respect to others and was translated into robot bowing movement. The experimental results indicated that human-like movement did not significantly affect trust repair, while constant-speed and abrupt movements showed improvements in subjective trust. The study highlighted the need for movement designs with considerations of the robot's appearance to facilitate effective trust repair.

Understanding how different levels of robotic assistance influence human perception, trust, and prosocial behavior is critical in human-robot interaction (HRI) research. This study investigates how the cost of help provided by a robot affects human perception, trust, and prosocial behavior by presenting participants with a video-based experiment. In the experiment, participants observed a humanoid robot, Sota, providing assistance under two conditions: high-cost help, where the robot shared power from its own battery, and low-cost help, where the robot facilitated power transfer from an external mobile battery. Results showed that participants perceived the robot as more anthropomorphic and intelligent in the high-cost condition, with increased trust ratings in both performance and moral trust dimensions. However, no significant difference was observed in participants’ prosocial behavior towards the robot. These findings suggest that while higher-cost robotic assistance enhances perception and trust, it does not necessarily lead to greater prosocial responses from humans. This study contributes to the broader understanding of how varying levels of robotic assistance impact human social responses and has implications for designing socially interactive robots in cooperative settings.

Trust, the cornerstone of human-robot interaction, is a key element in fostering a synergistic human-robot relationship. Trust facilitates the appropriate utilization of these systems, thereby optimizing their potential benefits. A failure to appropriately gauge the level of trust in a robot can have grave consequences, including potential misuse and accidents, underscoring the critical importance of accurate trust assessment in fostering a harmonious and safe human-robot collaboration. To avert such issues, it is imperative to calibrate trust levels accurately. To address this need, we have developed a novel estimation model for trust, leveraging the capabilities of structural equation modeling (SEM) to address the challenges posed by latent variables. The proposed model demonstrated a 70% accuracy in estimating trust during a manipulator’s successful and failed behaviors with uncertainty. The outcomes demonstrate the efficacy of the proposed method in surpassing conventional approaches.

As AI and robots become increasingly integrated into daily life, fostering trust in robots is essential for establishing long-term human-robot relationships. Enhancing people’s trust in robots can help mitigate anxiety and aversion toward them. While previous research has primarily focused on trust in robots based on their performance and achievements, this study explores the impact of robots appearing to trust humans on human decision-making. In this study, a robot performed the Prisoner’s Dilemma task three times with participants. We examined whether participants’ choices in the game were influenced by the robot’s eye color (blue, red), the robot’s behavior (available, not available), and before/after the task using a three-factor mixed design. The first experiment assessed whether the robot appeared to trust participants using a questionnaire. The second measured participants’ trust in the robot after the interaction. Analysis results indicated that as the number of Prisoner’s Dilemma interactions increased, participants were more likely to betray the robot. However, trust in the robot increased after the task, suggesting that participants felt more trusted by the robot, which in turn enhanced their own trust in it. This study introduces a novel perspective on human-robot relationships, highlighting how making people feel trusted by a robot can foster greater trust toward it.

Countdowns and count-ups are very useful displays that explicitly show how long users should wait and also show the current processing states of a given task. Most countdowns or count-ups decrease or increase their digit every one second exactly, and most users have an implicit assumption that the digit changes every one second exactly. However, there are no studies that investigate how users perceive wait times with these countdowns and count-ups and that consider changing users’ perception of time passing as shorter than the actual passage of time by means of countdowns and count-ups while taking into account such user assumptions. To clarify these issues, we first investigated how users perceive countdowns “from 3/5/10 to 0” and count-ups “from 0 to 3/5/10” that have different lengths of intervals from 800 to 1200 msec (Experiment 1). Next, on the basis of the results of Experiment 1, we explored a novel method for presenting countdowns to make users perceive the wait time as being shorter than the actual wait time (Experiment 2) and investigated whether such countdowns can be used in realistic applications or not (Experiment 3). As a result, we found that countdowns and count-ups that were “from 250 msec shorter to 10% longer” than 3, 5, or 10 sec were perceived as 3, 5, or 10 sec, respectively, and those “from 5 to 0” (their lengths were 5 sec) that first displayed extremely shorter intervals were perceived as being shorter than their actual length (5 sec). Finally, we confirmed the applicability and effectiveness of such displays in a realistic application. Thus, we strongly argue that these findings could become indispensable knowledge for researchers in this research field to reduce users’ cognitive load during wait times.

Communication robots have the potential to contribute to effective human-XAI interaction as an interface that goes beyond textual or graphical explanations. However, it is not clear how we can develop an adaptive strategy to use a robot’s physical and vocal expressions depending on the context in dynamic interactions. This paper proposes a method for a communication robot to decide where to emphasize XAI-generated explanations with physical expressions. In the method, a user model predicts the effect of emphasizing certain points on a user and aims to minimize the expected difference between predicted user decisions and AI-suggested ones. We conducted a user study to investigate how emphasis selection with our method affects the performance of user decisions. The results suggest that our method guides a part of users to better decisions when the performance of the AI suggestion is high.

This study investigated whether robotic agents that deal with social hierarchical relationships can reduce the dominance of superiors and equalize participation among participants in discussions with hierarchical structures. Thirty doctors and students having hierarchical relationship were gathered as participants, and an intervention experiment was conducted using a robot that can encourage participants to speak depending on social hierarchy. These were compared with strategies that intervened equally for all participants without considering hierarchy and with a no-action. The robots performed follow actions, showing backchanneling to speech, and encourage actions, prompting speech from members with less speaking time, on the basis of the hierarchical relationships among group members to equalize participation. The experimental results revealed that the robot’s actions could potentially influence the speaking time among members, but it could not be conclusively stated that there were significant differences between the robot’s action conditions. However, the results suggested that it might be possible to influence speaking time without decreasing the satisfaction of superiors. This indicates that in discussion scenarios where experienced superiors are likely to dominate, controlling the robot’s backchanneling behavior could potentially suppress dominance and equalize participation among group members.

Explainable AI (XAI) has been developed to make AI understandable to humans by providing explanations of its outputs. However, multiple types of AI explanations displayed on a screen could distort users’ trust in AI and their decision to rely on it. This could lead to poor task performance. In this study, we experimentally investigated the influence of AI explanations on trust and acceptance of AI results using a visual task. As a result, we found that participants increased their trust and acceptance of AI results with multiple types of explanations even though this did not improve task performance. These results, showing over-trust and over-reliance in human-agent interaction, were discussed along with cognitive load and cognitive bias caused by XAI.

This paper describes how user engagement affects their trust in audio guide agents to adapt to their environment. Audio guide agents are expected to be used more in the future because they can perform without interrupting the user’s actions, even when the user is in action. Because agents adapt to their environment and users work collaboratively with each other, trust in the agent is important. Therefore, we examined whether adaptation itself affects trust and whether intentional user operations as a means of user engagement affect trust. A participant experiment was conducted with two independent variables, the degree of adaptation to ambient noise and the availability of user operation, and the subjective evaluation of the impression of the agent as the dependent variable. For the factor of adaptability, agents with low noise cancellation performance and agents with high noise cancellation performance were provided. For the factor of user operation, we set whether or not to provide the user with volume adjustment functions for ambient noise. Regarding the agent’s impression, it was verified that users perceive the agent as more intelligent and human-like only when the agent’s adaptation to the environment is low. However adaptation of the agent could increase trust in the agent, and trust was also increased by intentional user operations regardless of the agent’s adaptation level. In systems where users and agents work collaboratively, the results suggest that intentionally increasing user engagement may increase trust in the agent.

As AI technology develops, the relationship between people and agents is becoming increasingly important as more agents are used in human society. One way to improve this relationship is to increase empathy for the agent. In this study, to increase empathy toward the agent, an agent was envisioned that assists participants in reflecting on traffic safety. Focusing on the agent’s attitude and behavior modification, three hypotheses were investigated through experimental testing. The results of experiment showed an interaction between the agent’s behavior modification and the before/after factors, indicating that a positive behavior modification maintains empathy for the agent. This research reveals an approach that promotes the social use of agents by people, which is necessary for the social coexistence of people and agents.

Service robots are now common in restaurant food delivery. However, machines aren’t always stable and may encounter errors, causing a loss of trust. Typically, when people make an error, they apologize and are forgiven. We were curious about how people react when a service robot makes an error and apologizes. Before checking trust in a participant, we aimed to check for manipulative behavior and investigate which motions are suitable for apologetic gesture such as bowing, using non-anthropomorphic robot. We conducted a web-based experiment with two-way ANOVA using a 2 x 5 (End-effector: Robot hand, Human hand; Motions: 5 different motions) within-participant design. Participants indicated a willingness to accept machine apologies, but we explored how trust could change. This study contributes insights into human-robot interactions, probing the acceptance of service robots in various roles and the impact of error and apology on trust.

This paper addresses the challenge of selecting explanations for XAI (Explainable AI)-based Intelligent Decision Support Systems (IDSSs). IDSSs have shown promise in improving user decisions through XAI-generated explanations along with AI predictions, and the development of XAI made it possible to generate a variety of such explanations. However, how IDSSs should select explanations to enhance user decision-making remains an open question. This paper proposes X-Selector, a method for selectively presenting XAI explanations. It enables IDSSs to strategically guide users to an AI-suggested decision by predicting the impact of different combinations of explanations on a user’s decision and selecting the combination that is expected to minimize the discrepancy between an AI suggestion and a user decision. We compared the efficacy of X-Selector with two naive strategies (all possible explanations and explanations only for the most likely prediction) and two baselines (no explanation and no AI support) in virtual stock-trading support scenarios. The results suggest the potential of X-Selector to guide users to AI-suggested decisions and improve task performance under the condition of a high AI accuracy.

This study investigated how a robot head motion towards an AI attention heatmap during a visual identification task influences a human user’s trust in eXplainable AI (XAI). The findings revealed that the robot head motion presented in a video increased the user’s acceptance of AI-generated results compared to the robot eye gaze displayed in a static image with or without the AI attention heatmap. However, displaying the heatmap improved task performance more than displaying no heatmap with or without the robot. Overall, these results suggest a possibility that showing a robot head motion towards an AI attention heatmap in a movie can serve as an interpretable XAI for visual tasks.

As anthropomorphic agents (AI and robots) are increasingly used in society, empathy and trust between people and agents are becoming increasingly important. A better understanding of agents by people will help to improve the problems caused by the future use of agents in society. In the past, there has been a focus on the importance of self-disclosure and the relationship between agents and humans in their interactions. In this study, we focused on the attributes of self-disclosure and the relationship between agents and people. An experiment was conducted to investigate hypotheses on trust and empathy with agents through six attributes of self-disclosure (opinions and attitudes, hobbies, work, money, personality, and body) and through competitive and cooperative relationships before a robotic agent performs a joint task. The experiment consisted of two between-participant factors: six levels of self-disclosure attributes and two levels of relationship with the agent. The results showed that the two factors had no effect on trust in the agent, but there was statistical significance for the attribute of self-disclosure regarding a person’s empathy toward the agent. In addition, statistical significance was found regarding the agent’s ability to empathize with a person as perceived by the person only in the case where the type of relationship, competitive or cooperative, was presented. The results of this study could lead to an effective method for building relationships with agents, which are increasingly used in society.

This study investigated how displaying an AI attention heatmap influences human acceptance of the AI’s suggestions in accordance with the interpretability of the heatmap. We conducted an experiment using a visual task where the participants were required to decide whether to accept or reject an AI’s suggestions. The participants could see the suggestions with an AI attention heatmap, the heatmap with the AI pointing to it (displayed as a laser dot cursor), the heatmap with a robot pointing (a robot using a stick to point to the AI heatmap displayed on a tablet), or no heatmap. The experimental results revealed that human acceptance of AI suggestions differed depending on the interpretability of the heatmap, especially when the heatmap was displayed with AI pointing. Also, additional analysis revealed an effect on acceptance due to the AI pointing to the heatmap that was found only in a high-task difficulty situation. An AI pointing to its attention heatmap is considered to trigger people to reason about particular AI processes and accept its suggestions. This study showed that an AI pointing to its attention heatmap could be used to control human behaviors in human-agent interaction.

Transparency in decision-making is an important factor for AI-driven autonomous systems to be trusted and relied on by users. Studies in the field of visual information processing typically attempt to make an AI system's behavior transparent by showing bounding boxes or heatmaps as explanations. However, it has also been found that an excessive amount of explanations sometimes causes information overload and brings negative results. This paper proposes SmartBBox, a method for reducing the number of bounding boxes to show while maintaining human reliance on an AI. It infers if each bounding box is worth showing by predicting its effect on human reliance. SmartBBox can autonomously learn to decide whether to show bounding boxes from humans' usage data. We implemented and tested SmartBBox in an autonomous driving scenario in which a human continuously decides whether to rely on an autonomous driving system while observing the dynamic results of object detection by the system. The results suggest that SmartBBox can reduce bounding boxes 64.8% on average from object recognition results while keeping human reliance at the same level as in the case where all the bounding boxes are presented.

Perspective-taking, which enables individuals to consider the thoughts and objectives of another, is well established to be a successful strategy for encouraging pro-social behavior in human-computer interactions. Nowadays, perspective-taking is no longer limited to text; it is now more frequently used in virtual reality (VR). However, most previous research has focused on simulating human-human interactions in the real world in VR by providing participants with experiences connected to different moral tasks. In this study, we investigated whether participants’ prosocial behaviors toward robots would change if they experienced an altruistic VR task involving robots from the perspective of different robots. Our findings show that participants who had the help-receiver-view exhibited more altruistic behaviors toward a robot than those who had the help-provider-view one in a dictator game. We believe that this work is the first attempt to investigate the relationship between perspective-taking in a VR environment and changes in prosocial behavior in human-robot interaction.

This study investigated how displaying a robot’s attention heatmap while the robot pointing gesture at it influences human trust and acceptance of its outputs. We conducted an experiment using two types of visual tasks. In these tasks, the participants were required to decide whether to accept or reject the answers of an AI or robot. The participants could see the answers with an AI attention heatmap, the heatmap with AI pointing (displayed as a laser dot cursor), a robot attention heatmap with robot pointing (pointing at a certain location on the heatmap displayed on a tablet with a stick), or no heatmap. The experimental results revealed that the AI and robot pointing at their attention heatmaps lowered the participants’ acceptance of their answers when the heatmaps had low interpretability in a more difficult task. Also, the robot pointing at the heatmaps showed the possibility of increasing acceptance of its answer when the heatmaps had high interpretability in a more difficult task. In addition, the acceptance of the robot’s answers correlated with emotional trust in the robot. This study demonstrates that a robot pointing gesture at its attention heatmap could be used to control human behaviors and emotional trust in human-robot interactions.

Abstract

This study investigated whether human trust in a social robot with anthropomorphic physicality is similar to that in an AI agent or in a human in order to clarify how anthropomorphic physicality influences human trust in an agent. We conducted an online experiment using two types of cognitive tasks, calculation and emotion recognition tasks, where participants answered after referring to the answers of an AI agent, a human, or a social robot. During the experiment, the participants rated their trust levels in their partners. As a result, trust in the social robot was basically neither similar to that in the AI agent nor in the human and instead settled between them. The results showed a possibility that manipulating anthropomorphic features would help assist human users in appropriately calibrating trust in an agent.

One way to improve the relationship between humans and anthropomorphic agents is to have humans empathize with the agents. In this study, we focused on a task between agents and humans. We experimentally investigated hypotheses stating that task difficulty and task content facilitate human empathy. The experiment was a two-way analysis of variance (ANOVA) with four conditions: task difficulty (high, low) and task content (competitive, cooperative). The results showed no main effect for the task content factor and a significant main effect for the task difficulty factor. In addition, pre-task empathy toward the agent decreased after the task. The ANOVA showed that one category of empathy toward the agent increased when the task difficulty was higher than when it was lower. This indicated that this category of empathy was more likely to be affected by the task. The task itself used can be an important factor when manipulating each category of empathy.

Today, people are enjoying using teleconference systems like Zoom or Skype for social communication or for having drinks through a screen with others via the internet. In addition, some people have started using embodied systems called telepresence robots, such as the Beam robot. Some schools have started using telepresence robots so that students can attend school. However, in previous studies, systems have not been compared in terms of social presence and anthropomorphism, for example, robots compared with humans. Therefore, we wondered how the presence and anthropomorphism of such systems affect people. Therefore, we carried out a web-based experiment and conducted a one-way ANOVA (smartphone vs. telepresence robot with motion vs. without motion). Some people feel that telepresence robots bring a feeling of presence to remote places. Ironically, from the results, a video teleconference system using a smartphone and a telepresence robot did not create a feeling of presence, but regarding anthropomorphism, participants felt more of a human-likeness in the video teleconference system.

Reinforcement learning, which acquires a policy maximizing long-term rewards, has been actively studied. Unfortunately, this learning type is too slow and difficult to use in practical situations because the state-action space becomes huge in real environments. The essential factor for learning efficiency is rewards. Potential-based reward shaping is a basic method for enriching rewards. This method is required to define a specific real-value function called a “potential function” for every domain. It is often difficult to represent the potential function directly. SARSA-RS learns the potential function and acquires it. However, SARSA-RS can only be applied to the simple environment. The bottleneck of this method is the aggregation of states to make abstract states since it is almost impossible for designers to build an aggregation function for all states. We propose a trajectory aggregation that uses subgoal series. This method dynamically aggregates states in an episode during trial and error with only the subgoal series and subgoal identification function. It makes designer effort minimal and the application to environments with high-dimensional observations possible. We obtained subgoal series from participants for experiments. We conducted the experiments in three domains, four-rooms(discrete states and discrete actions), pinball(continuous and discrete), and picking(both continuous). We compared our method with a baseline reinforcement learning algorithm and other subgoal-based methods, including random subgoal and naive subgoal-based reward shaping. As a result, our reward shaping outperformed all other methods in learning efficiency.

Although the processing speed of computers has been drastically increasing year by year, users still have to wait for computers to complete tasks or to respond. To cope with this, several studies have proposed presenting certain visual information to users to change their perception of time passing as shorter, e.g., progress bars with animated ribbing or faster/slower virtual clocks. As speech interfaces such as smart speakers are becoming popular, a novel method is required to make users perceive the passing of time as shorter by presenting auditory stimuli. We thus prepared 20 pieces of auditory information as experimental stimuli; that is, 11 auditory stimuli that have the same 10.1-second duration but different numbers of 0.1-second sine-wave sounds and 9 other auditory stimuli that have the same 10.1-second duration and numbers of sounds but different interval patterns between the sounds. We conducted three experiments to figure out which kinds of auditory stimuli can change users' perception of time passing as shorter. We found that a 10.1-second auditory stimulus that has 0.1-second sine-wave sounds appearing 11 times with intervals between the sounds that narrow rapidly in a linear fashion was perceived as shortest at about 9.3 seconds, which was 7.6% shorter than the actual duration of the stimulus. We also found that different interval patterns of sounds in auditory information significantly affected users' perception of time passing as shorter, while different numbers of sounds did not.

One of the most important problems in science education is teaching about the risks associated with pseudoscience. In this research, we focused on virtual teachers (VTs) that give lessons on pseudoscience. In prior research, the effect of robot teachers in scientific education was demonstrated, and the appearance of VTs was an important factor. Also, it was shown that the effect of logical persuasion and emotional persuasion changed on the basis of the context. Thus, we hypothesize that both the appearance of VTs and their persuasion strategy significantly affects the effect of the lessons through the interaction of the two. To verify this hypothesis, we conducted two-factor and two-levels experiments. One factor was the appearance of the VT: human-like or robot-like. Another was the persuasion strategy: emotional persuasion or logical one. As a result, a significant interaction was shown between the perceived persuasiveness of the VTs and their appearance. When the topic was minus ions' positive effect, the robotlike VT expressing emotional persuasion was perceived as less persuasive than the other VT. However, when the topic was UFOs, the robot-like VT expressing logical persuasion was perceived as less persuasive than the robot-like VT expressing emotional persuasion.

Trust calibration is essential to successful cooperation between humans and autonomous systems such as those for self-driving cars and autonomous drones. If users over-estimate the capability of autonomous systems, over-trust occurs, and the users rely on the systems even in situations in which they could outperform the systems. On the contrary, if users under-estimate the capability of a system, undertrust occurs, and they tend not to use the system. Since both situations hamper cooperation in terms of safety and efficiency, it would be highly desirable to have a mechanism that facilitates users in keeping the appropriate level of trust in autonomous systems. In this paper, we first propose an adaptive trust calibration framework that can detect over/under-trust from users' behaviors and encourage them to keep the appropriate trust level in a "continuous" cooperative task. Then, we conduct experiments to evaluate our method with semi-automatic drone navigation. In experiments, we introduce ABA situations of weather conditions to investigate our method in bidirectional trust changes. The results show that our method adaptively detected trust changes and encouraged users to calibrate their trust in a continuous cooperative task. We believe that the findings of this study will contribute to better user-interface designs for collaborative systems.

Using robots and virtual agents as teachers in education is one of the most important fields in HAI. Many pieces of work have been published; however, little has been reported on the relationship between the subject on which a virtual teacher (VT) gives a lesson and the appearance of the VT. For example, are robot-like agents usually effective regardless of the subject being taught? In this paper, we hypothesized that the subject and the appearance of the VT affect students' level of understanding through the interaction of the two. To verify this, we conducted an experiment with two factors: subject and VT appearance. Under all conditions, the participants watched movies in which VTs gave a lesson, and they took a short test on the lesson and provided a subjective evaluation on the VTs. As a result, the subject and appearance affected the short-test scores through the interaction of the two. This result suggests a novel design method that can be used to construct a VT.

Artificial subtle expressions (ASEs) are machine-like expressions used to convey a system's confidence level to users intuitively. So far, auditory ASEs using beep sounds, visual ASEs using LEDs, and motion ASEs using robot movements have been implemented and shown to be effective. In this paper, we propose a novel type of ASE that uses vibration (vibrational ASEs). We implemented the vibrational ASEs on a smartphone and conducted experiments to confirm whether they can convey a system's confidence level to users in the same way as the other types of ASEs. The results clearly showed that vibrational ASEs were able to accurately and intuitively convey the designed confidence level to participants, demonstrating that ASEs can be applied in a variety of applications in real environments.

In this work, we developed virtual humans (VH) designed to persuade users. We introduce the notion of distinctiveness of topics and define two kinds of persuasion strategies: "objective persuasion", which aims to persuade with only objective sentences and no expression, and "subjective persuasion", which aims to persuade with only subjective sentences and smiles and gestures. We performed experiments in which a VH recommended trips under two conditions. In condition 1, the VH recommended a topic that she preferred using subjective persuasion and another topic using objective persuasion. In condition 2, she recommended a topic that she preferred using objective persuasion and another topic using subjective persuasion. Results showed the VH in condition 1 increased the buying motivation of participants.

In this paper, we explore how a utility robot might express emotions via expressive lights and in-situ motions. In most previous work, methods for either modality were investigated alone, leaving a huge potential to improve the expression of emotions by combining the two modalities. We present a series of three studies, one for investigating how well people might recognize emotions on the basis of expressive light cues alone, one for exploring how people might perceive affect towards in-situ motion characteristics, and one for further combining the two modalities and studying whether multi-modal expressions could be better recognized by people. Results from the first study show participants were not able to recognize target emotions with high accuracy. Results from the second suggest a relationship between the in-situ motion characteristics of a robot and perceived affect. Results from the third suggest that expressions that combine in-situ motions with expressive lights were better able to convey many emotions but not all. We conclude that adding in-situ motions to affective expressive lights appears to be better able to help convey emotions. These findings are important for designing affective behaviors for future utility robots that need to possess certain social abilities.

Bioluminescence is the production and emission of light by a living organism. It, as a means of communication, is of importance for the survival of various creatures. Inspired by bioluminescent light behaviors, we explore the design of expressive lights and evaluate the effect of such expressions on a human»s perception of and attitude toward an appearance-constrained robot. Such robots are in urgent need of finding effective ways to present themselves and communicate their intentions due to a lack of social expressivity. We particularly focus on the expression of attractiveness and hostility because a robot would need to be able to attract or keep away human users in practical human-robot interaction (HRI) scenarios. In this work, we installed an LED lighting system on a Roomba robot and conducted a series of two experiments. We first worked through a structured approach to determine the best light expression designs for the robot to show attractiveness and hostility. This resulted in four recommended light expressions. Further, we performed a verification study to examine the effectiveness of such light expressions in a typical HRI context. On the basis of the findings, we offer design guidelines for expressive lights that HRI researchers and practitioners could readily employ.

This paper focused on the appearance of humanoid robot and their trustworthy and emotion richness perceived. Humanoid robots that used in emotional labor is needed to express emotion and be trusted. We experimented with eight robots image (four mechanical face robots and four smooth face robots) and asked the participants their impression. We conducted explanatory factor analysis to define the factors of robots' impression. As a result, the factors of robots were discovered to be different from the virtual humans' impression. Also, the trustworthy and emotion richness perceived of robots depended on another factors. The familiar robots were trusted and the human-like robots were expected to have rich emotion.

Functional robots are generally restricted in appearance, thus lacking ways to express their intent. In human-human interaction, gaze is an important cue for providing information and regulating interaction. In this pilot study, we investigate how we can implement gaze behavior in functional robots since gaze communication can allow humans to read a robot's intent and adjust their behavior accordingly. We explore design principles based on LED lights as we consider LEDs to be easily installed in most robots while not introducing features that are too human-like (to prevent users from having high expectations). In the paper, we present a design interface that allows designers to explore the parameter space of an LED strip attached to a Roomba robot. We then summarize a set of design principles for optimally simulating light-based gazes. Finally, our suggested design is evaluated by a large group of participants, and their comments are discussed.

Artificial subtle expressions (ASEs) are machine-like expressions used to convey a system's confidence level to users intuitively. In this paper, we focus on the cognitive loads of users in interpreting ASEs in this study. Specifically, we assume that a shorter response time indicates less cognitive load, and we hypothesize that users will show a shorter response time when interpreting ASEs compared with speech sounds. We succeeded in verifying our hypothesis in a web-based investigation done to comprehend participants' cognitive loads by measuring their response times in interpreting ASEs and speeches.

In this paper, we verified two kinds of two-dimensional mind perception models of humanoid virtual agents and investigate the relationship between the models and effect of emotional contagion. To verify the two kinds of dimensional models, we used questionnaires from prior works and our own questionnaire. From these questionnaires, we constructed an "agency"- "experience" model and "familiarity"-"'reality" model from EFA. These two models are valid for distinguishing humanoid agents and predicting the effect of emotional contagion. The factor scores of "experience" and "familiarity" have a high correlation coefficient with the effect of emotional contagion. This result suggests a method for designing humanoid agents that have a high emotional contagion ability.

Expressive light has been explored in a handful of previous studies as a means for robots, especially appearance- constrained robots that are not able to employ human-like expressions, to convey internal states and interact with people. However, it is still unknown how different light expressions can affect a person's perception and behavior. In this poster, we explore this research question by studying the effects of different expressive light animations on people's intention to buy hedonic products and how they choose between healthy and unhealthy food. Our preliminary results show that participants assigned to a positive and low arousal light animation condition had a higher intention of purchasing hedonic products and were inclined to choose unhealthy over healthy food. Such findings are in line with previous literature in marketing research, suggesting that mental alertness mediates the effect of external stimuli on a person's behavioral intentions. Future work is thus required to evaluate such findings in a human-robot interaction context.

Many researchers are now dedicating their efforts to studying interactive modalities such as facial expressions, natural language, and gestures. This phenomenon makes communication between robots and individuals become more natural. However, many robots currently in use are appearance constrained and not able to perform facial expressions and gestures. In addition, although humanoid-oriented techniques are promising, they are time and cost consuming, which leads to many technical difficulties in most research studies. To increase interactive efficiency and decrease costs, we alternatively focus on three interaction modalities and their combinations, namely color, sound, and vibration. We conduct a structured study to evaluate the effects of the three modalities on a human's emotional perception towards our simple-shaped robot "Maru." Our findings offer insights into human-robot affective interactions, which can be particularly useful for appearance-constrained social robots. The contribution of this work is not so much the explicit parameter settings but rather deepening the understanding of how to express emotions through the simple modalities of color, sound, and vibration while providing a set of recommended expressions that HRI researchers and practitioners could readily employ.

We conducted three experiments to discover the effect of a virtual agent's state transition on a user's eye gaze. Many previous studies showed that an agent's state transition affects a user's state. We focused on two kinds of transitions, the internal state transition and appearance state transition. In this research, we used a product recommendation virtual agent (PRVA) and aimed to discover the effect of its state transitions on users' eye gaze as it made recommendations. We used entropy-based analysis to visualise the deviation of a user's fixations. In experiment 1, the PRVA made recommendations without state transitions. In experiment 2, the amount of the PRVA's knowledge transitioned from low to high during the recommendations. This is an internal state transition. In experiment 3, the PRVA's facial expressions and gestures transitioned from a neutral to positive emotion during the recommendations. This is an appearance state transition. As a result, both the entropy-based analysis and fixation duration based analysis showed significant differences in experiment 3. These results show that an agent's appearance state transitions cause a user's eye gaze to transition.

A preliminary experiment is carried out to investigate the effects of LED light animations on a user's perception of a computer. As anthropomorphism has become an important factor in interaction design, current research tends to add human-like expression abilities to interactive devices. Such methods, however, have limitations as they are complex and not applicable to many currently-in-use appearance-constrained devices such as personal computers. Thus, in this work we investigate an alternative method: expressive light. We attached a programmable RGB LED strip to the front-bottom of a monitor and developed a ping pong game for carrying out an experiment. We collected both game log and questionnaire data from participants. Our results show that participants who played the game with LED light animations liked the game more and perceived the computer as better and more humanlike. In addition, no evidence suggested a negative effect on a user's task performance or lead to additional workload.

Natural language user interfaces, such as Apple Siri and Google Voice Search have been embedded in consumer devices; however, speaking to objects can feel awkward. Use of these interfaces should feel natural, like speaking to a real listener. This paper proposes a method for manufactured objects such as anime figures to exhibit highly realistic behavioral expressions to improve speech interaction between a user and an object. Using a projection mapping technique, an anime figure provides back-channel feedback to a user by appearing to nod or shake its head.

In this research, we aim to suggest a method for designing trustworthy PRVAs (product recommendation virtual agents). We define an agent's trustworthiness as being operated by user emotion and knowledgeableness perceived by humans. Also, we suggest a user inner state transition model for increasing trust. To increase trust, we aim to cause user emotion to transition to positive by using emotional contagion and to cause user knowledgeableness perceived to become higher by increasing an agent's knowledge. We carried out two experiments to inspect this model. In experiment 1, the PRVAs recommended package tours and became highly knowledgeable in the latter half of ten recommendations. In experiment 2, the PRVAs recommended the same package tours and expressed a positive emotion in the latter half. As a result, participants' inner states transitioned as we expected, and it was proved that this model was valuable for PRVA recommendation.

As robotics has advanced, research on conveying a robot's emotional state to a person has become a hot topic. Most current studies are focused on interaction modalities such as facial expressions and natural language. Although many of the results seem to be promising, they suffer from high cost and technical difficulties. In this paper, we turn our attention to three other interaction modalities: color, sound, and vibration. Such modalities have the advantage of being simple, low cost, and intuitive. We conducted a pilot study to evaluate the effects of the three modalities on a human's emotional perception towards our robot Maru. Our result indicates that humans tend to interpret a robot's emotion as negative (angry in particular) when vibration and sound are used, while they interpret the emotion as relaxed when only color modality is used. In addition, the participants showed preference towards the robot when using all three modalities.

The purpose of this work is to investigate which of an agent's properties determines leader-follower relationships in cooperative tasks performed by a human and an agent (a computer). The possible factors of an agent are intelligence, obstinance, and appearance. In this paper, we focused on intelligence and obstinance and conducted a psychological experiment using a mark matching game with a declaration phase, which enables us to observe who becomes the leader in a cooperative task. Experimental results showed that humans tend to follow an agent who has low intelligence and more obstinance rather than an agent who has high intelligence and less obstinance, and we found that obstinance is more important than intelligence in being a leader in human-computer interaction.

The notion of Emotional contagion is a phenomenon in which a human emotion infects others. Various studies have been done to cause emotional contagion between a human and a robot or an anthropomorphic agent in HRI research fields. However, few studies have been done to compare different kinds of agents in order to find important properties in emotional contagion in human-agent interaction (HAI). In this research, we conducted an experiment to determine which properties cause this phenomenon between anthropomorphic agents and users. We prepared two kinds of agents. One is a cartoon-like agent, and the other is a concrete agent. The cartoon-like agent smiled exaggeratedly, and the concrete agent smiled modestly. As a result, we found that the concrete agent was more effective than the cartoon-like agent at emotional contagion. This result suggests a model for designing more trustworthy and familiar agents and robots.

We propose a model for designing embodied virtual agents for use in online retailing. Such agents are necessary for transiting user internal states to manipulate the decision-making process. To satisfy this demand, the agents need to build a trustworthy relationship with users through interactions. In our model, user internal state transitions are derived from agent state transitions. The trustworthiness of agents changes according to agent states and is difficult to operate directly. We define two factors of trustworthiness, emotion and intelligence, and propose a way to transit these two factors of the agents to improve their trustworthiness.

Human-interactive robots are assessed according to various factors, such as behavior, appearance, and quality of interaction. In the present study, we investigated the hypothesis that impressions of an unattractive robot will be improved by emotional interaction with physical touch with the robot. An experiment with human subjects confirmed that the evaluations of the intimacy factor of unattractive robots were improved after two minutes of physical and emotional interaction with such robots.

The use of mobile devices that utilize touch panels as interfaces, such as smartphones and tablet PCs, has spread in recent years, and these have many advantages. For example, panels can be operated more intuitively than those with conventional physical buttons, and the devices are much more flexible than those that use traditional fixed UIs. However, mistakes frequently occur when inputting with a touch panel because the buttons have no physical boundaries and users cannot get tactile feedback from their fingers. Thus, the input accuracy of touch-panel devices is lower than that of devices with physical buttons. There have been studies on improving input accuracy. Most of them have used language models for typing natural language or probabilistic models to describe the errors made when users tap panels with their fingers. However, these models are not practical because they deal with kinematic errors, not cognitive errors. Thus, we propose a more practical model for improving input accuracy in this paper, in which the tap model includes cognitive errors to avoid tapping neighboring objects to a target object. We consider that our model can describe important properties for designing various UIs depending on practical applications. We also conducted experiments to build our model in a calibrated way and discussed our evaluation of the model and revision of the model.

In recent years, devices that use touch panels as interfaces, such as smart phones and tablet PCs, have spread. These devices have many advantages. For example, operating the panel can be done more intuitively in comparison with using conventional physical buttons, and the devices are quite more flexible than those that use a traditional fixed UI. However, mistakes frequently occur when inputting with a touch panel because the buttons have no physical boundaries and users cannot get tactile feedback with their fingers because the panels never change physically. Thus, the input accuracy of touch-panel devices is lower than that of devices with physical buttons. There are studies on improving input accuracy. Most of them use language models for typing natural language or probabilistic models to describe the errors made when users tap their fingers. However, these models are not practical, and the experiments are preliminary. Thus, in this paper, we propose a more practical model for improving input accuracy, in which the relative relationships between a target object and neighbor objects that might influence error making when touching the target are tested. We consider that our model can describe important properties for designing various UIs depending on practical applications. We also conducted preliminary experiments in order to build our model in a calibrated way and discuss our evaluation of the model.

We conducted an experimental investigation on human adaptation to change in an agent's strategy through a competitive two-player game. Modeling the process of human adaptation to agents is important for designing intelligent interface agents and adaptive user interfaces that learn a user's preferences and behavior strategy. However, few studies on human adaptation to such an agent have been done. We propose a human adaptation model for a two-player game. We prepared an on-line experimental system in which a participant and an agent play a repeated penny-matching game with a bonus round. We then conducted experiments in which different opponent agents (human or robot) change their strategy during the game. The experimental results indicated that, as expected, there is an adaptation phase when a human is confronted with a change in the opponent agent's strategy, and adaptation is faster when a human is competing with robot than with another human.

A method for creating a constrained clustering ensemble by learning the priorities of pair wise constraints is proposed in this paper. This method integrates multiple clusters produced by using a simple constrained K-means algorithm that we modify to utilize the constraints priorities. The cluster ensemble is executed according to a boosting framework, which adaptively learns the constraints priorities and provides them for the modified constrained K-means to create diverse clusters that finally improve the clustering performance. The experimental results show that our proposed method outperforms the original constrained K-means and is comparable to several state-of-the-art constrained clustering methods.

The impressions made by a blinking light used to create artificial subtle expressions (ASEs) and by a robot's appearance on users were investigated. The blinking light, which shows the user that the robot is performing speech recognition and thereby prevents utterance collisions, was separated from the robot by embedding it in a pedestal unit. In an evaluation experiment, participants performed five tasks with a spoken dialogue system coupled to a robot placed on the pedestal. The participants' impressions of the dialogue interactions and of the robot were obtained under four conditions (w/ light blinking or w/o blinking; humanoid or cuboid robot). The cuboid robot created a stronger impression of comfort and excitement for the interactions while the blinking light did not create a strong impression of anything. The robot's appearance and the blinking did not create a strong impression of anything for the robot. This suggests that the blinking light in the pedestal unit is a factor that is independent of robot appearance, meaning that the pedestal unit can be applied to robots with various appearances.

The Turing test is an imitation game for determining the intelligence of an agent. In spite of its simplified setting, the use of natural language between two agents in the test is still too high a hurdle for achieving fruitful results in the field of artificial intelligence. In this paper, the authors propose a variation of the Turing test with a restricted communication method. This modified test uses behaviors generated by two-axis actuators for communication instead of the natural language dialogue used in the normal Turing test. This reduction of scope reveals what kinds of features are essential for an imitation game, and broaden the application brought by Turing test. When we learn what sorts of communication become possible with restricted actuation, we can apply this knowledge to any kind of robot or device in the real world. First, we tried to determine what elements are critical for communication between a user and a robot through a preliminary experiment involving human-human communication. A human manipulator received a video image as input and controlled a "robot box" with two actuators in a way that would lead a user to put other objects into the box. The results indicated what kinds of behavior are required to show the intention of the manipulator to the user. Second, we analyzed the result of the preliminary experiment, organized a behavioral model from the result, and programmed the robot box to run the model. The behavior of the robot was programmed according to the user's head and hand locations as identified by a motion captures system. The robot automatically interact with a human without human manipulation with this program. Third, we conducted a behavioral Turing test in a communication task whereby the human collected items according to the instructions of the robot box. In this test, two actuators on the box is controlled both by human manipulator and our program. The answers of users suggests that the users could not identify which is controlled by a human manipulator or the program. This result indicates that the Turing test succeed in a restricted behavioral level.

A user working at his/her desktop computer would benefit from notifications being given at timings that reflect their relevancy to the user's activity and workload. To do so correctly, a notification system should have a way of determining the user's state of activity We propose a novel method to estimate user states with a pressure sensor on a desk. We use a lattice-like pressure sensor sheet and distinguish between two simple user states: busy or idle. The pressure can be measured without the user being aware of it, and changes in the pressure reflect useful information like typing, an arm, the presence of a coffee mug, and so on. We carefully developed features which can be extracted from the sensed data and used a machine learning technique to identify the user state. We conducted experiments evaluating the accuracy of our method and obtained promising results.

Auditory instruction is a well used method for people of all ages because of its understandability. However the additional voice has the possibility to disturb the user's learning during the instruction because it strongly implies the support of third-person helpers. This risk increases with older people because their confidence in their ability may decline compared to the younger people. The authors propose a method to anthropomorphize an instructed target (a vacuum) to decrease the feeling of a third person during instruction. The authors conducted the experiment using our method to explain features of household appliance and evaluated the relationship between recalled features and older people's internal scale. The results show that older people remembered more features by using our method, and with female participants, their internal scales increased during the training. This demonstrates that our method can decrease the third-person feeling in female participants and increase the amount learned. Our findings suggest that auditory instructions may be an effective learning method for older adults.

In this study, we propose a disembodied real-world agent and the study of the influence of this disembodiment on the social separation between the user and the agent. In order to give a clue to the user about the presence of the robot and to make possible a visual feedback, we decide to use independent robotic body parts that mimic human hands and eyes. This robot is also able to share real-world space with the user, and react to his presence, through 3d detection and oral communication. Thus, we can obtain an agent with an important presence while keeping good space efficiency, and as a result ban any existing social barrier.

Users' impressions of blinking light expressions used as artificial subtle expressions have been investigated. In a preliminary experiment, thirteen blinking patterns were used for investigating participants' impressions of their agreeableness. The highest and lowest valued blinking patterns were identified and used for a speech interaction experiment. In this experiment, 52 participants tried to reserve hotel rooms with a spoken dialogue system coupled with an interface robot using a blinking light expression. A sine wave, a random wave, a rectangular wave, and a no-blinking condition were used as artificial subtle expressions to express a robot's internal state of “processing” or “recognizing”. The results of a questionnaire showed the conditions did not significantly differ in terms of agreeableness, but the sine wave and the rectangular wave were evaluated as “more useful” than the no-blinking condition. Results of factor analyses suggested that the rectangular wave provides a comfortable impression of the dialogue.

Unpredictable user behaviors in a physical process represent one of the fundamental obstacles to the realization of a Cyber-Physical System. In this paper, we propose the use of social cues such as body shape, expressions, and verbal timing to control user behaviors in the physical world. Social cues can control user behaviors in both their spatial and temporal aspects. As a result, user actions become more predictable in a CPS. We consider how social cues restrict user behaviors by referring to a number of psychological, cognitive, and human-robot interaction studies, and we propose a model of restriction based on social cues. Using this model, we created hardware and software in order to realize attachable social cues, and we seek to demonstrate the effect of social cues using the example of home appliances.

An experimental investigation into how the appearance of an agent such as a robot or PC affects people's interpretations of the agent's attitudes is presented. In general, people are said to create stereotypical agent behavioral models in their minds based on the agents' appearances, and these appearances significantly affect their way of interaction. Therefore, it is quite important to address with the following research question: How does an agent's appearance affect its interactions with people? Specifically, a preliminary experiment was conducted to select eight artificial sounds for which people can estimate two specific primitive attitudes (e.g., positive or negative). Then an experiment was conducted where the participants were presented with the selected artificial sounds through three kinds of agents: a MindStorms robot, AIBO robot, and laptop PC. In particular, the participants were asked to select the correct attitudes based on the sounds expressed by these three agents. The results showed that the participants had better interpretation rates when a PC presented the sounds and lower rates when the MindStorms and AIBO robots presented the sounds, even though the sounds expressed by these agents were the same. The results of this study contribute to the design policy of the interactive agents, such as, What types of appearances should agents have to effectively interact with people, and which kinds of information should these agents express to people?

We describe artificial subtle expressions (ASEs) as intuitive notification methodology for artifacts' internal states for users. We prepared two types of audio ASEs; one was a flat artificial sound (flat ASE), and the other was a sound that decreased in pitch (decreasing ASE). These two ASEs were played after a robot made a suggestion to the users. Specifically, we expected that the decreasing ASE would inform users of the robot's lower level of confidence about the suggestions. We then conducted a simple experiment to observe whether the participants accepted or rejected the robot's suggestion in terms of the ASEs. The results showed that they accepted the robot's suggestion when the flat ASE was used, whereas they rejected it when the decreasing ASE was used. Therefore, we found that the ASEs succeeded in conveying the robot's internal state to the users accurately and intuitively.

In this paper, we compare user behavior towards the humanoid robot ASIMO and the dog-shaped robot AIBO in a simple task, in which the users has to teach commands and feedback to the robot.

Our study compares users' interaction with a humanoid robot and a dog-shaped pet-robot. We conducted a user study in which the participants had to teach object names as well as simple commands to either the humanoid or the pet-robot and give feedback to the robot for correct and incorrect performance. While we found, that the way of uttering commands rather depends on personal preference than on the robots' appearance, the way of giving positive and negative feedback differed significantly between both robots: We found that for the pet-robot users gave reward in a similar way as giving reward to a real dog by touching it and commenting on its performance by uttering feedback like “well done” or “that was right”. For the humanoid, users typically did not use touch as a reward and rather used personal expressions like “thank you” to praise the robot. Our findings suggest that users actually rely to some degree on the appearance of a robot as a cue for deciding how to interact with it.

This paper describes user studies on a novel remote control manipulatable with stroking its surface. There are lots of remote controls in our houses such as remote controls for TV, air conditioner, and so on. However, when we use a remote control, we need to look at both our fingers and an appliance that we would like to control. It may be not significantly problematic for young people, but elderly people have a difficulty in manipulating remote controls with many buttons. We consider it will be comfortable for various people to use a remote control without looking at their fingers and pushing buttons. We also consider a remote control should have robot-like appearance to become a more familiar artifact to users. In this study, we have proposed a robot-like remote control, Rebo, manipulatable only with stroking its surface and apply to an advanced TV system. The developed remote control has three advantages; familiarity, function awareness, and stroke manipulation, in contrast with conventional remote controls with many buttons. These advantages enable users to feel much familiarity by using it, to easily notice its implemented functions, and to use it without looking at the fingers and buttons. In this paper, we focus on experimental investigation for advantages of Rebo. We conducted experiments with participants and the experimental results supported such advantages.

Hierarchical menus are now widely used as standard user interfaces in modern applications with GUIs. The menu performance depends on many factors, such as the structure, layout, and colors. There has been extensive research on novel hierarchical menus, but there has been little work on improving performance by optimizing the menu's structure. We have proposed an algorithm based on a genetic algorithm (GA) for optimizing the performance of menus. The algorithm aims to minimize the average selection time of menu items by taking into account movement and decision-making time. We have shown that the proposed algorithm can reduce average selection time nearly 40% for a menu of a cellar phone. But usage pattern were limited and the accuracy of the model was not confirmed. We will first show the validation result of the model by experiments conducted on PDA. Then we will present results of the performance evaluation of the algorithm by using a wide variety of usage patterns generated by Zipf function. The results show that the model has good accuracy for real users, and the algorithm can attain good results for a wide variety of usage patterns.

We propose a method to enable a robot to learn simple, parameterized commands, such as ldquoPlease switch on the TV!rdquo or ldquoCan you bring me a coffee?ldquo for human-robot interaction. The robot learns through natural interaction with a user in a special training task. The goal of the training phase is to allow the user to give commands to a robot in his preferred way instead of learning predefined commands from a handbook. Learning is done in two successive steps. First the robot learns object names. Then it uses the known object names to learn parameterized command patterns and determine the position of parameters in a spoken command. The algorithm uses a combination of hidden Markov models and classical conditioning to handle alternative ways to utter the same command and integrate information from different modalities.

This paper describes a new remote control operable with stroking its surface. There are lots of remote controls in our houses such as TV remote controls, air conditioner remote controls, and so on. However, when we use a remote control, we need to look at both the fingers and an appliance that we want to control. It is not highly problematic for young people, but elderly people have a difficulty in operating remote controls. It will be comfortable for people to use a remote control without looking at the fingers. In this study, we propose a remote control, Rebo, operable with stroking its surface and apply to a TV interaction system. The developed remote control has three advantages; (1) familiarity, (2) function awareness, and (3) stroke operation. Those enable users to have familiarity with it, to easily notice its functions, and to use it without looking at the fingers. The feature of Rebo in comparison with conventional button-based remote controls is the tolerance for mistakes because it enables unfamiliar users to home electric appliances to use it casually without fear of mistakes and unexpected behavior.

We are proposing an approach to enable a robot to learn the speech, gesture and touch patterns, that its user employs for giving positive and negative reward The learning procedure uses a combination of Hidden Markov Models and a mathematical model of classical conditioning. To facilitate learning, the robot and the user go through a training task where the goal is known, so that the robot can anticipate its user’s commands and rewards. We outline the experimental framework and the training task and give details on the proposed learning method evaluating the applicability of classical conditioning for the task of learning user rewards given in one or more modalities, such as speech, gesture or physical interaction.

An experimental investigation of how the appearance of agents affects interpretations people make of the agents’ attitudes is described. We conducted a psychological experiment where participants were presented artificial sounds that can make people estimate specific agents’ primitive attitudes from three kinds of agents, e.g., Mindstorms robot, AIBO robot, and a normal laptop PC. Specifically, the participants were asked to select the appropriate attitude based on the sounds expressed by these three agents. The results showed that the participants had higher correct interpretation rates when a PC presented the sounds, while they had lower rates when Mindstorms and AIBO robots presented the sounds, even though these agents expressed information that was completely the same.

We propose a framework to extract topic maps from a set of Web pages. We use the clustering method with the Web pages and extract the topic map prototypes. We introduced the following two points to the existing clustering method: The first is merging only the linked Web pages, thus extracting the underlying relationships between the topics. The second is introducing weighting based on the similarity from the contents of the Web pages and relevance between topics of pages. The relevance is based on the types of links with directories in the Web sites structure and the distance between the directories in which the pages are located. We generate the topic map prototypes by assuming that the clusters are the topics, the edges are the associations, and the Web pages related to the topics are the occurrences from the results of the clustering. Finally, users complete the prototype by labeling the topics and associations and removing the unnecessary items. We incrementally use a user’s evaluation of the topic maps to judge whether a Web page is unnecessary or necessary and then reduce the number of unnecessary pages. We use the relevance feedback along with a Support Vector Machine (SVM) to judge the Web pages. For this paper, at the first step, we mounted the proposed clustering method and conducted experiments to evaluate the effectiveness of extracting topic map prototypes. We eventually discussed the effectiveness of our two additional points by evaluating the extracted topic map prototypes.

Hierarchical menus are widely used as a standard user interface in modern applications that use GUIs. The performance of the menu depends on many factors: structure, layout, colors and so on. There has been extensive research on novel menus, but there has been little work on improving performance by optimizing the menu’s structure. This paper proposes algorithms based on the genetic algorithm (GA) and the simulated annealing (SA) for optimizing the performance of menus. The algorithms aim to minimize the average selection time of menu items by considering the user’s pointer movement and search/decision time. We will show the results on a static hierarchical menu of a cellular phone as an example where a small screen and limited input device are assumed. We will also show performance comparison of GA-based algorithm and the SA-based one by using wide variety of the useage patterns.

Hierarchical menus are now ubiquitous. The performance of the menu depends on many factors: structure, layout, colors and so on. There has been extensive research on novel menus, but there has been little work on improving the performance by optimizing the menu's structure. This paper proposes an algorithm based on the genetic algorithm (GA) for optimizing the performance of menus. The algorithm aims to minimize the average selection time of menu items by considering movement and decision time. We show results on a static hierarchical menu of a cellular phone where a small screen and limited input device are assumed. Our work makes several contributions: a novel mathematical optimization model for hierarchical menus; novel optimization method based on the genetic algorithm (GA).

Hierarchical menus are now ubiquitous. The performance of the menu depends on many factors: structure, layout, colors and so on. There has been extensive research on novel menus, but there has been little work on improving performance by optimizing the menu's structure. This paper proposes algorithms based on the genetic algorithm (GA) and the simulated annealing (SA) for optimizing the performance of menus. The algorithms aim to minimize the average selection time of menu items by considering the user's pointer movement and search/decision time. We will show the experimental results on a static hierarchical menu of a cellular phone as an example where a small screen and limited input device are assumed. We will also show performance comparison of the GA-based algorithm and the SA-based one by using wide varieties of usage patterns.

Speech overlaps, undesired collisions of utterances between systems and users, harm smooth communication and degrade the usability of systems. We propose a method to enable smooth speech interactions between a user and a robot, which enables subtle expressions by the robot in the form of a blinking LED attached to its chest. In concrete terms, we show that, by blinking an LED from the end of the user's speech until the robot's speech, the number of undesirable repetitions, which are responsible for speech overlaps, decreases, while that of desirable repetitions increases. In experiments, participants played a last-and-first game with the robot. The experimental results suggest that the blinking-light can prevent speech overlaps between a user and a robot, speed up dialogues, and improve user's impressions.

This paper describes an experimental study in which we analyze how users give multimodal positive and negative feedback by speech, gesture and touch when teaching easy game-tasks to a pet robot. The tasks are designed to allow the robot to freely explore and provoke human reward behavior. By choosing game-based tasks, we ensure that the training can be carried out without stressing or boring the user. This way, we can observe natural, situated reward behavior.

We propose a method to enable smooth speech interactions between a user and a robot. Our method is based on subtle expression whereby a robot blinks a small LED attached to its chest. We performed experiments in which participants played a last-and-first games and counted the number of repetitions made by the participants and analyzed their impression of the game and the robot. The experimental results suggested that the blinking-light could prevent utterance collisions between a user and a robot and could create familiar and attentive impressions about the game on users.

The Job-Shop Scheduling Problem (JSSP) is well known as one of the most difficult NP-hard combinatorial optimization problems. Several GA-based approaches have been reported for the JSSP Among them, there is a parameter-free genetic algorithm (PfGA) for JSSP proposed by Matsui et al., based on an extended version of PfGA, which uses random keys for representing permutation of operations in jobs, and uses a hybrid scheduling for decoding a permutation into a schedule. They reported that their algorithm performs well for typical benchmark problems, but the experiments were limited to a small number of problem instances. This paper shows the results of an empirical performance evaluation of the GA for a wider range of problem instances. The results show that the GA performs well for many problem instances, and the performance can be improved greatly by increasing the number of subpopulations in the parallel distributed version.

This paper proposes a novel method of learning a users preferred reward modalities for human-robot interaction through solving a cooperative training task. A learning algorithm based on a combination of adaptable pre-trained hidden Markov models and a computational model of classical conditioning is outlined. In a training task, where the desired outcome is known by an AIBO pet robot as well as its human instructor, the robot can freely explore human reward behavior. By this method, the robot is able to learn situated, user-specific reward behavior in the different modalities such as gestures, speech and interaction using the robot's built-in sensors. After the training phase, the learned reward behavior can be used as a basis for reinforcement learning of more complex tasks. A preliminary experimental study is presented, which investigates on the effects of restricting possible reward modalities, when teaching a pet robot. The results of the experiments suggest that being able to provide reward freely makes users give more reward compared to a scenario, where reward modalities are restricted. Moreover, the experiments showed that even if a restriction in possible reward modalities is introduced, users tend to give reward that does not conform to the restriction.

This paper focuses on the problem that will arise in the near future from multi-function robots. Users will have to read thick operation manuals to use them. If users can use these robots without reading difficult manuals, it will improve user efficiency. We then proposed action sloping as a way for users to naturally recognize a robot's function. It provides the robots with gradual feedback signals when the user performs given actions. By changing the intensity of the feedback signal according to his/her action, it encourages him/her to perform an action that will trigger the robot's function. In our experiments, we made three kinds of feedback behaviors according to Action Sloping and one non-feedback behavior as the control condition. The participants of the experiment tried to find a robot's function and the latencies to first finding the triggered action were measured. An analysis of the latencies show ed the difference between the sound feedback group by action sloping and the control group. This result showed that the effectiveness of action sloping was partially supported.

In late years, as new media turning into PC or mobile phone, a study about communication robots is prosperous. Robots are different from the conventional media, because robots have physical bodies like humans, so it is reported that humans bemust robots socially. Therefore, we decide to apply a concept of meme as a cultural gene to an interaction design with humans and robots in this study. By this design theory, we will realize Mimetic Mutual Adaptation by humans and robots imitating and adapting each other, exceeding a conventional form of oneway adaptation from humans to the media. Therefore, we called cultural information transmitted from robots "RobotMeme", we try that robots acquire cultural behaviors shared by human society and the robots transmits these meme to other robots by Human-Robot Mimetic Mutual Adaptation. Furthermore, we suggest "A Design of RobotMeme" to realize that robots create new cultural behaviors through human-robot interaction. In this paper, we describe an early stage of experiments to inspect whether RobotMeme were transmitted to human and observed that humans acquired original cultural behaviors of robots by imitation. From the results of these experiments, it is suggested that robots and human are going to be able to form the relations of interdependence by imitating each other.

This paper describes experimental investigation how agents' appearances affect users' interpretations of agents' attitudes. Specifically, we conducted a psychological experiment that participants were presented artificial sounds as subtle expressions that can make human estimate specific agents' primitive attitudes from three kinds of different agents, e.g., Mindstonns robot, AIBO robot, and normal laptop PC, and they were asked to select the correct attitudes based on the expressed sounds from these three agents. As the result, the participants showed the higher interpretation rates when the sounds were presented from PC, while they did the lower rates from Mindstorms and AIBO robots, even though the artificial sounds expressed from these agents were the completely same sounds. Finally, the result was compared to the Mori's assumption that is about the relationship between an agents' appearance (likeness) and users' familiarity to the agents.

This paper describes our efforts to investigate factors in user's browsing behavior to automatically evaluate web pages that the user shows interest in. To evaluate web pages automatically, we developed a client-side logging/analyzing tool: the GINIS Framework. This work focuses primarily on client-side user behavior using a customized web browser and AJAX technologies. First, GINIS unobtrusively gathers logs of user behavior through the user.s natural interaction with the web browser. Then it analyses the logs and extracts effective rules to evaluate web pages using C4.5 machine learning system. Eventually, GINIS becomes able to automatically evaluate web pages using these learned rules.

Query expansion is an information retrieval technique in which new query terms are selected to improve search performance. Although useful terms can be extracted from documents whose relevance is already known, it is difficult to get enough of such feedback from a user in actual use. We propose a query expansion method that performs well even if a user makes practically minimum effort, that is, chooses only a single relevant document. To improve searches in these conditions, we made two refinements to a well-known query expansion method. One uses transductive learning to obtain pseudorelevant documents, thereby increasing the total number of source documents from which expansion terms can be extracted. The other is a modified parameter estimation method that aggregates the predictions of multiple learning trials to sort candidate terms for expansion by importance. Experimental results show that our method outperforms traditional methods and is comparable to a state-of-the-art method.

This paper describes an application of SVM (support vector machines) to interactive document retrieval using active document showing. Some works have been done to apply classification learning like SVM to relevance feedback and obtained successful results. However they did not fully utilize characteristic of example distribution in document retrieval. We propose heuristics to bias document showing according to distribution of examples in document retrieval. This heuristic is executed by selecting examples to show a user in neighbors of positive support vectors, and it improves learning efficiency. We implemented a SVM-based interactive document retrieval system using our proposed heuristic, and compare it with conventional systems like Rocchio-based system and a SVM-based system without the heuristic. We conducted systematic experiments using large data sets including over 500,000 paper articles and confirmed our system outperformed other ones

This paper reports a new document retrieval method using non-relevant documents. Especially, this paper reports a comparison of retrieval efficiency between one class support vector machine (SVM) based and support vector data description (SVDD) based interactive document retrieval method using non-relevant documents only. From a large data set of documents, we need to find documents that relate to human interesting in as few iterations of human testing or checking as possible. In each iteration a comparatively small batch of documents is evaluated for relating to the human interesting. We applied active learning techniques based on support vector machine for evaluating successive batches, which is called relevance feedback. Our proposed approach has been very useful for document retrieval with relevance feedback experimentally. The traditional relevance feedback needs a set of relevant and non-relevant documents to work usefully. However, the initial retrieved documents, which are displayed to a user, sometimes don't include relevant documents. In order to solve this problem, we propose a new feedback method using information of non-relevant documents only. We named this method non-relevance feedback document retrieval. The non-relevance feedback document retrievals are based on one class support vector machine and support vector data description. Our experimental results show that one class support vector machine based method can retrieve relevant documents efficiently using information of non-relevant documents only.

This paper describes designing expression of robot's primitive minds in a simple and effective way. Although expression of robot's minds like internal states is becoming a major topic in human-robot interaction, few studies to propose a policy to design concrete expression. In this paper, we propose the design policy, called "SE2PM: simple expression to primitive mind", and fully implement expression of robot's minds based on the design policy, then investigate the effectiveness of them. SE2PM implies that intuitive and simple expression like a beep sound from a simple robot (e.g. a mobile robot) is more effective than complicated behaviors from a complex robot (e.g. a dog-like pet robot) in informing its primitive minds to a human. Thus, in order to investigate the validation of our SE2PM policy, we implement two robots: a mobile robot based on Mindstorms with mind expression of beep sound, and a pet robot, AIBO, with mind expression of complicated behaviors. We also conduct a psychological experiment with participants to compare the two different expression, and the results eventually support SE2PM policy

This paper describes our efforts to factor in a user's browsing behavior to automatically evaluate web pages that the user shows interest in, based on user browsing behaviors while browsing. To evaluate a webpage automatically, we have developed a client-side logging tool: the GINIS Framework. We do not focus just on clicking, scrolling, navigation, or duration of visit alone, but we propose integrating these patterns of interaction to recognize and evaluate a user's response to a given web page.

This paper reports a new document retrieval method using non-relevant documents. From a large data set of documents, we need to find documents that relate to human interesting in as few iterations of human testing or checking as possible. In each iteration, a comparatively small batch of documents is evaluated for relating to the human interesting. We applied active learning techniques based on support vector machine for evaluating successive hatches, which is called relevance feedback. Our proposed approach has been very useful for document retrieval with relevance feedback experimentally. The relevance feedback needs a set of relevant and non-relevant documents to work usefully. However, the initial retrieved documents, which are displayed to a user, sometimes don't include relevant documents. In order to solve this problem, we propose a new feedback method using information of non-relevant documents only. We named this method non-relevance feedback document retrieval. The non-relevance feedback document retrieval is based on one-class support vector machine. Our experimental results show that this method can retrieve relevant documents using information of non-relevant documents only.

In this paper, we propose a novel interaction model for a human-robot cooperative task. The CEA (Commands Embedded in Actions) model reduces a human work-load because a user of a robot needs less inputs and outputs than DCM (Direct Commanding methods) like gesturing. We propose ECEA (Extended CEA) in order to deal with more complicated tasks than CEA. On the cooperative sweeping task between a human and a mobile robot, we apply temporal extension as one of ECEA instances. Multiple commands are embedded in the human action by the extension and the robot performs more complicated task. The experiments for conforming reduction of a human work-load using ECEA are conduced on the sweeping task. Human cognitive loads are measured as human workloads and compared between ECEA and DCM. The results of the experiments showed that the ECEA minimized a human cognitive load.

We investigate the following data mining problems from the document retrieval: From a large data set of documents, we need to find documents that relate to human interest as few iterations of human testing or checking as possible. In each iteration a comparatively small batch of documents is evaluated for relating to the human interest. We apply active learning techniques based on support vector machine for evaluating successive batches, which is called relevance feedback. Our proposed approach has been very useful for document retrieval with relevance feedback experimentally. In this paper, we adopt several representations of the vector space model and several selecting rules of displayed documents at each iteration, and then show the comparison results of the effectiveness for the document retrieval in these several situations.

In this paper, we propose a Future View system that assists user's usual Web browsing. A Future View prefetches Web pages based on user's browsing strategies and present them to a user in order to assist Web browsing. To learn browsing patterns for a user, Future View uses two types of learning classifier systems: a content-based classifier system for contents change patterns and an action-based classifier system for user's action patterns. The results of learning are applied to crawling by Web robot, and gathered Web pages are presented to a user through a Web browser. We experimentally show the effectiveness of navigation using a Future View.

We investigate the following data mining problems from the document retrieval: From a large data set of documents, we need to find documents that relate to human interesting in as few iterations of human testing or checking as possible. In each iteration a comparatively small batch of documents is evaluated for relating to the human interesting. We apply active learning techniques based on Support Vector Machine for evaluating successive batches, which is called relevance feedback. Finally, our proposed approach is very useful for document retrieval with relevance feedback experimentally.

We propose a future view system that assists user's usual Web browsing. The future view will prefetch Web pages based on user's browsing strategies and present them to a user in order to assist Web browsing. To learn user's browsing patterns, the future view uses two types of learning classifier systems: a content-based classifier system for contents change patterns and an action-based classifier system for user's action patterns. The results of learning are applied to crawling by Web robot, and gathered Web pages are presented to a user through a Web browser. We experimentally show effectiveness of navigation using the future view.

In this paper, we describe influence of viewpoints of observation in an interactive evolutionary robotics system. We have been proposed a behavior learning system ICS (Interactive Classifier System) using interactive evolutionary computation. In this system, a mobile robot is able to quickly learn rules by direct teaching of a human operator. ICS is a novel evolutionary robotics approach using a classifier system. We classify teaching methods into internal observation and external one, and investigate influence of observation methods. We have experiments based on our teaching methods in two kinds of tasks. We found that teaching methods from different viewpoints of observation change teaching efficiency because of the difference between a robot's recognition and an operator's one in an environment.

This paper describes an organization method of page information agents for adaptive interface between a user and a Web search engine. Though a Web search engine indicates a hit list of relevant Web pages, it includes many useless ones. Thus a user often needs to select useful Web pages from them with page information like the title, the URL on the hit list, and actually fetch the Web pages for checking relevance. Since the page information is neither sufficient nor necessary for a user, adequate information is necessary for valid selection. Hence we propose adaptive interface AOAI in which different page information agents are organized through human evaluation.

In this paper, we propose a [Formula: see text] (Personal Web Map) which is a personal and small database of interesting Web pages to a user and develop a method to construct it under the user's control of multiple Web robots. While general search engines with very large databases are valid for information retrieval in the WWW, it is still important that a user constructs a small, personal database of relevant Web pages to his/her interest. For such a Web page database, we propose a [Formula: see text] and develop a [Formula: see text] system. First a user gives keywords indicating his/her interest to a system, and it constructs a [Formula: see text] concerned with the keywords. For building a useful [Formula: see text], it is necessary that a user can interrupt the construction of a [Formula: see text] anytime and instruct a sub-field which should be explored more. For this function, we develop an anytime-control algorithm for multiple Web robots. A density blackboard is used for controlling Web robots, and an uniform distributed [Formula: see text] is built. Whenever a system is interrupted by a user, it provides a valid [Formula: see text] in terms of keeping search space wide, and indicates many alternatives on which he/she wants more information. From Web pages in a database, document vectors are generated and used to construct a 2D-map of a [Formula: see text] by using self-organization maps. A user easily recognizes interim results through the 2D-map, and gives instruction by clicking a node about which he/she wants more detail information. We made experiments by subjects and found out that our method outperformed breadth-first search for constructing a useful [Formula: see text]. As results, a [Formula: see text] system is considered as a promising approach to assist a user in gathering relevant information in the WWW.

The paper describes a novel action selection method for multiple mobile robots box-pushing in a dynamic environment. The robots are designed to need no explicit communication and be adaptive to dynamic environments by changing modules of behavior. The various control methods for a multirobot system have been studied both in centralized and decentralized approaches, however, they needed explicit communication such as a radio, though such communication is expensive and unstable. Furthermore, though it is a significant issue to develop adaptive action selection for a multirobot system to a dynamic environment, few studies have been done on it. Thus, we propose action selection without explicit communication for multirobot box-pushing which changes a suitable behavior set depending on a situation for adaptation to a dynamic environment. First, four situations are defined with two parameters: the existence of other robots and the task difficulty. Next, we propose an architecture of action selection which consists of a situation recognizer and sets of suitable behaviors to the situations and carefully design the suitable behaviors for each of the situations. Using the architecture, a mobile robot recognizes the current situation and activates the suitable behavior set to it. Then it acts with a behavior-based approach using the activated behaviors and can change the current situation when the environment changes. We fully implement our method on four real mobile robots and conduct various experiments in dynamic environments. As a result, we find out our approach is promising for designing adaptive multirobot box-pushing.

In this paper, we propose a PWM (Personal Web Map) which is a personal and small database of interesting Web pages to a user and develop a method to construct it under the user's control of multiple Web robots. Though general search engines with large databases like YaHoo, AltaVista, MetaCrawler are valid it is important that a user constructs a small, personal database of relevant Web pages to his/her interest like Bookmarks. For such a Web page database, we propose a PWM: a personal database of interesting Web pages to a user which he/she can control its construction. First a user gives keywords indicating his/her interest to a system, and it constructs a PWM concerned with the keywords. For building a useful PWM, it is necessary that a user can interrupt the construction of a PWM anytime and instruct a sub-field in which a PWM should be expanded more. For this function, we develop an anytime-control algorithm for multiple Web robots. A density distribution blackboard is used, and an uniform distributed PWM is built. Whenever a system is interrupted by a user it provides a valid PWM in terms of keeping search space wide, and indicates many alternatives on which he/she wants more information. From Web pages in a database, document vectors are generated and used to construct a 2D-map of a PWM by using self-organization maps. A user easily recognizes a PWM through the 2D-map, and gives instruction by clicking a node about which he/she wants more detail information. We made experiments by users and found out that our method outperformed breadth-first search for constructing a useful PWM. As results, a PWM system is considered as a promising approach to assist a user in gathering relevant information in the WWW.

Describes an action selection method for multiple mobile robots box-pushing in a dynamic environment. The robots are designed to need no explicit communication, and be adaptive to a dynamic environments by changing modules of behaviors. Though it is a significant problem to deal with adaptive action selection for multiple mobile-robots in a dynamic environment, few studies have been done. Decentralized control of robots without explicit communication is also practical and important for robustness. Thus we propose adaptive action selection without explicit communication for multi-robot box-pushing, which changes an available behavior set depending on a situation. First four situations are defined with two parameters: existence of other robots and task difficulty. Next we design a set of behaviors for each situation, and mobile robots are programmed to act with a behavior-based approach. We fully implement our method on four real mobile robots, and make experiments in dynamic environments.

We describe the development of a mobile robot which does unsupervised learning for recognizing environments from behavior sequences. Most studies on recognizing an environment have tried to build precise geometric maps with high sensitive and global sensors. However such precise and global information may not be obtained in real environments. Furthermore unsupervised-learning is necessary for recognition in unknown environments without help of a teacher. Thus we attempt to build a mobile robot which does unsupervised-learning to recognize environments with low sensitivity and local sensors. The mobile robot is behavior-based and does wall-following in enclosures. Then the sequences of behaviors executed in each enclosure are transformed into input vectors for a self-organizing network. Learning without a teacher is done, and the robot becomes able to identify enclosures. Moreover we developed a method to identify environments independent of a start point using a partial sequence. We have fully implemented the system with a real mobile robot, and made experiments for evaluating the ability. As a result, we found out that the environment recognition was done well and our method was adaptive to noisy environments.

Pauses play important roles both for the intelligibility and the naturalness of speech. Their occurrences and durations in text reading are influenced by syntactic structures of the text as well as by physiological constraints of respiration on the part of the speaker. The present paper describes some of the preliminary findings on Japanese text reading, especially on the effects of the syntactic role of the preceding phrase on the rate of occurrence and the duration of a pause at a syntactic boundary.

The authors describe a method to interleave planning with execution in the dynamic world. The system interleaves planning with execution using the success probability, SP: likelihood that a plan is executed well. Elemental probabilities are assigned to operations' effects and irrelevant state descriptions. A partial plan is transformed into a Bayesian network and its SP is computed. An agent switches planning to execution when SP decreases less than an execution threshold. The authors made experiments in Tileworld and found optimal thresholds between reactivity and deliberation. This is a preliminary report for solving tradeoff between reactivity and deliberation.

This new consulting system with graphical interfaces can assist a naive user in disassembling and reassembling a cylindrical machine by analyzing the structural description. Most trouble shooting systems developed so far do not tell us the way for decomposing the object to find out trouble points. If we could not know how to disassemble the object, we would be unable to inspect the trouble point. This system can assist naive users in both disassembling and reassembling an object. A user can hardly disassemble by simple command sequences that the system gave. This system automatically generates all possible procedures of disassembling the object from the 3D models. Then an integrated instruction facility using a visual interface must be offered for specifying what portion of the object should be dis/reassembled at the next stage, and for verifying whether user's operation is correct or not. And in future, a manipulator may dis/reassemble the machine instead of man. Furthermore we suggest the integrated maintenance system consisting of this consulting system and other modules.