Although Large Language Models (LLMs) have recently shown remarkable performance in many language comprehension tasks, they struggle to perform adequately in communicative contexts involving implicature. In our previous study, we proposed LLM-based agents by integrating LLMs with cognitive models. In three dialogue scenarios, these agents generated appropriate utterances as if inferring the speaker’s intentions (i.e., implicature). Further investigation of the agents’ performances requires an examination of their utterances in a large number of scenarios. In addition, it is also important to consistently evaluate the agents’ utterances. Thus, this study proposes a method in which LLMs evaluate agents’ generated utterances in the same way as human evaluators. Using our pilot prompt, we demonstrated that the evaluations of LLMs and human evaluators were similar.

Large language models are increasingly used as conversational agents but often follow user instructions uncritically, even when based on false beliefs. We introduce the DIVE protocol, a prompting strategy grounded in the Belief–Desire–Intention (BDI) framework, which guides models to Doubt the premise, Infer the user’s mental state, Verify belief consistency, and Expand the plan. We evaluated ChatGPT-4o, as an LLM target on ten story-based scenarios under four progressively structured prompting conditions. Success rates rose from 30% (baseline) to 40%, 70%, and 90% as BDI scaffolding and relations were added, indicating that the BDI structure can substantially improve belief-sensitive reasoning in LLMs.

Despite the significant development of Large Language Models (LLMs), they struggle with “dialogue communication based on implicature,” which humans handle easily. In this study, we aim to improve the performance of LLMs in this type of dialogue tasks by integrating LLMs with a cognitive model of dialogue. The cognitive model of dialogue comprises beliefs, desires, and intentions, which is defined as the Mental Model of Others (MMO) for predicting or estimating other’s mental states and behaviors. We propose two integration methods: the LLM Embedded in Cognitive Model (LEC) and the Cognitive Model Embedded in LLM (CEL). We examined the performances of our proposed method using the dialogue task, showing that the LEC can respond appropriately in the dialogues with implicatures, which cannot be achieved by conventional LLMs.

In the human-agent interaction (HAI) task, the mechanism underlying the human interpretation of logic is an important issue. However, despite the rapid development of sensory technologies, the human understanding algorithm is still at an elemental level, and novel methodologies are required. Thus, our aim was to develop an algorithm and a system that can estimate the human mental state by combining physical information from sensors and human knowledge about the interpretation of the behavior of others. As an initial step, we focused on the "interest" component of human behavior and demonstrated effectiveness at estimating human understanding. We then show the results for semi-automatic intention estimation in children. This system may open the way for HAI researchers to understand the real-world interactions that are driven by the human mental state.

We propose a technique to estimate a child's extraversion and agreeableness for social robots that interact with children. The proposed approach observed children's behavior using only the robot's sensors, without any sensor networks in the environment. An RGBD sensor was used to track and identify children's facial expressions. Children's interactions with the robot were observed, such as their distance from the robot and the duration of their eye contact, because such information would provide clues to estimate their personality. Data were collected when a robot, tele-operated by preschool teachers, interacted with kindergarten children individually. The data from 29 children was used to successfully estimate the children's personality compared to chance rates.

In this paper, we present a method of estimating a child's attention, one of the more important human mental states, in a free-play scenario of child-robot interaction. First, we developed a system that could sense a child's verbal and non- verbal multimodal signals such as gaze, facial expression, proximity, and so on. Then, the observed information was used to train a Support Vector Machine (SVM) to estimate a human's attention level. We investigated the accuracy of the proposed method by comparing with a human judge's estimation, and obtained some promising results which we discuss here.

Intimacy is a very important factor not only for the communication between humans but also for the communication between humans and robots. In this research we propose an action decision model based on others' friendliness value, which can be estimated using the mental models of others. We examine the mutual adaptation process of two agents, each of which has its own model of others, through the interaction between them.

Though communication is one of our basic activity, it is not always that we can interact effectively. It is well known that a key point for a successful interaction is the inclusion of other with a good mood. It means acquisition of other's interest is a precondition for a successful communication.

It is difficult to design robotic playmates for introverted children. Therefore, we examined how a robot should play with such shy children. In this study, we hypothesized and tested an effective play strategy for building a good relationship with shy children. We conducted an experiment with 5- to 6-year-old children and a humanoid robot teleoperated by a preschool teacher. We developed a valid play strategy for shy children.

Similar to relationships between humans, a person desiring to form a good relationship with a robot needs to be able to empathize with it. However, the specific kinds of human-robot interactions that would arouse and enhance empathy for the robot in the user's mind have not yet been clarified. In addition, the human behavioral traits that may be regarded as indices of empathy have not been investigated extensively. In an attempt to address these two issues, a preliminary experiment on empathy in human-robot interaction is conducted. The results suggest that the actions of naming or comforting a robot could contribute to enhancing its user's empathy and that eye fixation could be used as an index of empathy even when the use of a subjective index is inconclusive.

The influence of holding hands on the relationship building process between children and robots is investigated in this study. In particular, at the first meeting, it is difficult for a child to be open if he/she starts to rebuff the robot partner. This significantly degrades the possibility of the child forming a friendship relationship with the robot. Thus, the initial approach of the robot to the child in the early stage of the relationship building process should be appropriate. We hypothesize that physical embodied communication, such as walking hand in hand, improves the relationship between children and robots. A holding hands system was implemented in a real robot, and an experiment was conducted at a kindergarten to validate our hypothesis. The results strongly support our hypothesis.

Recently, we have mobile adaptive information terminal to navigate like to present information during walking (ex. smartphone and glasses-type wearable terminal). On the other hand, attention to the surrounding environment is insufficient and incident is a familiar occurrence. It is considered to solve such a problem, it is important to behavior support by by measuring human behavior in real time and feedback information intuitive. Human walking is affected by vision, vestibular, somatic and other various sensations that come through the sensory-motor loop. But detail of the sensory-motor loop is not clear. In this study, we examined a possible affect of self motion sensation by an optical-flow stimulus in peripheral vision with a decayed soma to-sensory feeling by a vibration stimulus on leg and foot area. In the experiment, we presented the optical flow for forward direction to the peripheral vision, and then gave the self-motion sensation by changing the flow to left or right direction. In this paper, we discuss on the unifying mechanism of visual and somatic sensations based on the experimental result.

We propose a playmate robot system that can play with a child. Unlike many therapeutic service robots, our proposed playmate system is implemented as a functionality of the domestic service robot with a high degree of freedom. This implies that the robot can play high-level games with children, i.e., beyond therapeutic play, using its physical features. The proposed system currently consists of ten play modules, including a chatbot with eye contact, card playing, and drawing. The algorithms of these modules are briefly discussed in this paper. To sustain the player's interest in the system, we also propose an action-selection strategy based on a transition model of the child's mental state. The robot can estimate the child's state and select an appropriate action in the course of play. A portion of the proposed algorithms was implemented on a real robot platform, and experiments were carried out to design and evaluate the proposed system.

We decide and execute our action from many types of environmental information in our daily lives even if we are not conscious of being guided. The human action induced from the oncoming person's movement is the collision avoidance of passing each other. In collision avoidance, we chiefly judge the avoidance direction from visual information. Especially, it is important to get the information from oncoming person's body part and avoidance timing in each other. Then, we make an experiment to judge the avoidance direction by watching the masking movie of oncoming person's body part. By evaluating this judgment time, it was clarified oncoming person's body part is leg in collision avoidance. Next, it especially paid attention to oncoming person's leg, and the relation between the walking cycle and leg position in avoidance judgment is evaluated. From this result, the avoidance judgment is possible because the traveling direction can be controlled by the leg when the leg is lifting and landing. It was clarified that oncoming person's walking cycle is important in the action decision during collision avoidance. So we propose the action decision model based at the walking cycle.

In social interactions human can behave cooperatively by estimating the intention of others and predicting their behavior. Understanding of others as having intentional states such as beliefs and desires is called Theory of Mind (ToM). This paper proposes a computational framework based on a Functional Parts Combination model (FPC model) for investigating the emergence of a ToM through adaptation in both evolutionary and individual-learning time scales. The FPC model is based on the neuroscientific fact that each cerebral cortical area has a different role and is selectively activated depending on the task. It consists of a set of functional parts and activation signals specifying selectively activated patterns. As a first step, this paper reports on the computer simulations which focused on the learning of the activation signals using a hunter task as a task to be solved by the agents. The simulation demonstrated a scenario for bootstrapping ToM as the emergence of the partial-networks of functional parts in the brain model based on the interactions between the levels of ToM.

We tried to guide human actions using galvanic vestibular stimulation (GVS), which might be a source of human behavior guidance without any attention. We tried to guide the trajectory of subject hands when they were continuously drawing circles. Previous work has mainly dealt with such unstable actions as walking and reaching in a standing posture. In this work, we verified the effects of GVS with a stable sitting posture under a head fixed condition in continuous circle drawing as a guided action. The results showed cases in which the hand is guided by GVS. From these experimental results, we hypothesize that GVS might trigger motion prediction for circle drawing. We confirmed the acceleration in the direction where a right and left imbalance arose when drawing a second circle after GVS. This suggests the possibility of a guiding hand trajectory and a triggered motion prediction by GVS in a stable posture where GVS cannot drastically change the balance perception.

We use various types of action decision strategies to realize smooth interaction with others. We can predict the intentions of others and determine our actions based on those predictions or induce the intentions and actions of others as we intended. In this study, we constructed a computational model of action decisions based on the intention predictions of others and evaluated its effectiveness by a computer simulation in a cooperative game. We also analyzed human behavior in the cooperative game. We suggest the necessity of meta-strategy such as a choice of action decision strategies from possibilities to determine the next action. We also discuss the feature of human meta-strategies for the action decision process in such interactive situations.

This paper presents a method for learning novel objects from audio-visual input. Objects are learned using out-of-vocabulary word segmentation and object extraction. The latter half of this paper is devoted to evaluations. We propose the use of a task adopted from the RoboCup@Home league as a standard evaluation for real world applications. We have implemented proposed method on a real humanoid robot and evaluated it through a task called “Supermarket”. The results reveal that our integrated system works well in the real application. In fact, our robot outperformed the maximum score obtained in RoboCup@Home 2009 competitions.

Performing general human behavior by experts' navigation is expected to be realized as wearable and ubiquitous technologies and computing develop. For simple, ordinary behavior, a person does not need the assistance of an expert. However, if one is standing next to an injured/ill person, one needs the instruction on performing first aid treatment from an expert. The wearer of the wearable behavior navigation system will be able to conduct first aid treatment as an expert would. We have developed the wearable behavior navigation systems using Augmented Reality technology, mainly for the navigation of the first aid treatment and for escape from dangerous areas, such as a building on fire. The effectiveness of the wearable navigation systems has been evaluated by a number of experiments. In this paper, the basic mechanism to realize general human behavior navigation is presented, along with the concrete configuration of the prototype of the navigation systems, and the experimental evaluation.

The capability to perform specific human tasks with the assistance of expert navigation is expected to be realized through the development wearable and ubiquitous computing technology. For instance, when an injured or ill person requires first-aid treatment, but only non-experts are nearby, instruction from an expert at a remote site is necessary. A behavior navigation system will allow the user to provide first-aid treatment in the same manner as an expert. Focusing on first-aid treatment, we have proposed and developed a prototype wearable behavior navigation system (WBNS) that uses augmented reality (AR) technology. This prototype WBNS has been evaluated in experiments, in which participants wore the prototype and successfully administered various first-aid treatments. Although the effectiveness of the WBNS has been confirmed, many challenges must be addressed to commercialize the system. The head-mounted displays (HMDs) used in the WBNS have a number of drawbacks, for example, high cost (which is not expected to decrease in the near future) and the time required for an ordinary user to become accustomed to the display. Furthermore, some individuals may experience motion sickness wearing the HMD. We expect that these drawbacks to the current technology will be resolved in the future; meanwhile, a near-future remote behavior navigation system (RBNS) is necessary. Accordingly, we have developed RBNSs for first-aid treatment using off-the-shelf components, in addition to the WBNS. In this paper, the basic mechanisms of the RBNS, experiments investigating the demonstration of expert behavior, and a comparative study of the WBNS and the RBNSs are presented.

In this study, we present a computational model of a car driver's cognitive process and eye movement in relation to risk evaluation for greater safe driving assistance. The model status is mainly determined by visual inputs. In simulations, we reconstructed and predicted the driver eye movements while driving using an environmental risk calculation model, and compared them to the eye movements of an actual human driver.

We tried to guide human action using galvanic vestibular stimulation (GVS). GVS has a possibility of human behavior guidance without any attention. We tried to guide the trajectory of the subjects' hands when as the continuously drew circles. Previous work has mainly dealt with unstable actions, such as walking and reaching in a standing posture. On the basis of the results, it was claimed that GVS is effective for human action guidance. However, in those experiments, GVS influenced just the perception of the direction of gravity direction and balancing. Clarifying the GVS effect for actions performed with stable postures is required. In this work, we verified the effects of GVS with a stable sitting posture under a head-fixed condition in continuous circle drawing as a guided action. The results showed that there are cases in which the hand is guided by GVS. This means that there is a possibility of guiding hand trajectory by GVS even with a stable posture where GVS cannot drastically change balance perception.

The task of variable selection is essential to improving the ability of machine learning systems to generalize. Although there are many conventional variable selection methods, almost all of them need to prepare and learn a large number of samples in advance because they are based on offline learning. This property is not suitable for online learning systems. To overcome this inconvenience, we propose a quick online variable selection method inspired by human problem solving behaviors. The proposed method tries to generate several variable set candidates in a speculative manner using a filter method and evaluates them using a wrapper method. The method can also function in concept-drifting environments, where relevant variable sets are changing. The experimental results show that the new method yields appropriate variable sets from a small number of samples.

This work presents a quick machine learning system inspired by human learning behavior. Data-mining systems based on machine-learning usually need a large number of iterations to acquire correct solutions, whereas people usually find appropriate hidden rules after only a small number of observations of the instances in a dataset. We think that this quick learning is the result of using tentative hypothesis as the data-model in the early steps of the learning. If the hypothesis happens to be accurate, the learning would be completed immediately after the hypothesis is applied. We therefore reduce the effort required for learning by gambling on the possibility that the tentative hypothesis is accurate. Our new machine learning system emulates this process by minimizing an objective function that represents not only the likelihood of error but also the predicted learning-cost. In experiments, the new system yields appropriate solutions to function approximation problems with only a small number of observations of instances. This system would be helpful for emergent problem solving.

We aim at online planning for real-time navigation of a mobile robot. To realize this, we consider map formation in an unknown environment and an online path search on the obtained map. In the map learning process, the symbol map is constructed by the allocation of a location symbol based on the discretization of the range sensor signal and the association of the neighboring relations to the location symbol. In addition, the misallocation of symbols caused by sensory aliasing is corrected based on the prediction by a neighboring relation. For the path search on the symbol map, the landmark view pattern is associated with the location symbol. We realized efficient path search by a method using the similarity of landmark view patterns.