Although Large Language Models (LLMs) have recently shown remarkable performance in many language comprehension tasks, they struggle to perform adequately in communicative contexts involving implicature. In our previous study, we proposed LLM-based agents by integrating LLMs with cognitive models. In three dialogue scenarios, these agents generated appropriate utterances as if inferring the speaker’s intentions (i.e., implicature). Further investigation of the agents’ performances requires an examination of their utterances in a large number of scenarios. In addition, it is also important to consistently evaluate the agents’ utterances. Thus, this study proposes a method in which LLMs evaluate agents’ generated utterances in the same way as human evaluators. Using our pilot prompt, we demonstrated that the evaluations of LLMs and human evaluators were similar.

Large language models are increasingly used as conversational agents but often follow user instructions uncritically, even when based on false beliefs. We introduce the DIVE protocol, a prompting strategy grounded in the Belief–Desire–Intention (BDI) framework, which guides models to Doubt the premise, Infer the user’s mental state, Verify belief consistency, and Expand the plan. We evaluated ChatGPT-4o, as an LLM target on ten story-based scenarios under four progressively structured prompting conditions. Success rates rose from 30% (baseline) to 40%, 70%, and 90% as BDI scaffolding and relations were added, indicating that the BDI structure can substantially improve belief-sensitive reasoning in LLMs.

Despite the significant development of Large Language Models (LLMs), they struggle with “dialogue communication based on implicature,” which humans handle easily. In this study, we aim to improve the performance of LLMs in this type of dialogue tasks by integrating LLMs with a cognitive model of dialogue. The cognitive model of dialogue comprises beliefs, desires, and intentions, which is defined as the Mental Model of Others (MMO) for predicting or estimating other’s mental states and behaviors. We propose two integration methods: the LLM Embedded in Cognitive Model (LEC) and the Cognitive Model Embedded in LLM (CEL). We examined the performances of our proposed method using the dialogue task, showing that the LEC can respond appropriately in the dialogues with implicatures, which cannot be achieved by conventional LLMs.

The spread of video streaming services has increased the opportunities for video watching, and research on co-watching with agents is also being conducted. However, few studies have been conducted and not insufficient knowledge has been obtained. In this study, we will conduct an investigation on the relationship among the background of the co-watchers, the video evaluation, and the agent’s impression. In the experiment, we asked the participants to co-watch a basketball game with an agent who responded appropriately then, and after watching, they answered a questionnaire. As a result, there was a moderate correlation between the Likeability and the evaluation of the video watching. There was also a correlation between the evaluation of the video watching and the empathy for the agent.

Most of agents that learn policy for tasks with reinforcement learning (RL) lack the ability to communicate with people, which makes human-agent collaboration challenging. We believe that, in order for RL agents to comprehend utterances from human colleagues, RL agents must infer the mental states that people attribute to them because people sometimes infer an interlocutor's mental states and communicate on the basis of this mental inference. This paper proposes PublicSelf model, which is a model of a person who infers how the person's own behavior appears to their colleagues. We implemented the PublicSelf model for an RL agent in a simulated environment and examined the inference of the model by comparing it with people's judgment. The results showed that the agent's intention that people attributed to the agent's movement was correctly inferred by the model in scenes where people could find certain intentionality from the agent's behavior.

In interactions, estimating how much information is being shared between participants is one of the crucial aspects that make the interaction more lively and enhance each participant's sense of understanding of the others. In this paper, we propose a model to estimate how much information is being shared between participants in a conversation. In the proposed model, we considered not only the content of the utterance but also the timing of the utterance. To verify the validity of our model, we implemented a simulator of a party game called Word Wolf, which requires information sharing estimation as the part of the game, and simulated the participants' behavior. Through the simulation, we showed that utterance timing is an important backchannel when estimating information sharing.

In research on semi-autonomous telepresence robots, a problem in which remote operators become frustrated with autonomous operations that do not match their intention has been reported. However, in previous research, a general-purpose method for automatically switching between remote and autonomous operations has not been proposed. In this paper, through the use of a general purpose arbitration model, called the accumulator based arbitration model (ABAM), we propose an adaptive switching architecture for remote and autonomous operations, named "One Minder." We incorporated One Minder into a semi-autonomous telepresence system autonomizing contingent behaviors, and conducted experiments to verify its utility using a robot implementing the proposed architecture. As the experiment results indicate, it was shown that One Minder can adaptively switch between remote and autonomous operations without manual switching. In addition, One Minder was also shown to reduce the operational load and frustration given to a remote operator by allowing the arbitration to properly output an autonomous operation.

Shared autonomy is a situation in which agents adapt to users based on feedbacks to jointly accomplish tasks. In the case of semiautonomous agents such as mobile robots operational load can be reduced by adaptively automating their behavior. Machine learning is one of the method for adapting teleoperated agents to users as control tendency depends on users and environments. It is difficult to regard user inputs as supervisions because user controls are not always available during descisionmaking processes for agents that continually make decisions (such as navigation robots).

In cooperation, the workers must know how co-workers behave. However, an agent's policy, which is embedded in a statistical machine learning model, is hard to understand, and requires much time and knowledge to comprehend. Therefore, it is difficult for people to predict the behavior of machine learning robots, which makes Human Robot Cooperation challenging. In this paper, we propose Instruction-based Behavior Explanation (IBE), a method to explain an autonomous agent's future behavior. In IBE, an agent can autonomously acquire the expressions to explain its own behavior by reusing the instructions given by a human expert to accelerate the learning of the agent's policy. IBE also enables a developmental agent, whose policy may change during the cooperation, to explain its own behavior with sufficient time granularity.