In the field of Human-Agent Interaction (HAI), many researchers have so far studied trait inference on artificial agents. However, most of them have used appearance and behavior as manipulation factors and have not directly manipulated the internal factors underlying the generation of agent behavior. To solve such issues, we argue the use of cognitive models as a tool to examine the process of participants’ trait inference on virtual agents. By using cognitive models that detail internal states, we can experimentally assume the link between behavior and cognitive characteristics. In this study, we use the Adaptive Control of Thought–Rational (ACT-R), which is a framework for constructing cognitive models, to construct a model and examine whether the properties of the model can be evaluated by humans. We focused on “curiosity,” a property of agents, and visualized the behavior of multiple cognitive models of curiosity as material for subjective ratings on agents’ traits. As a result, we confirmed that participants can estimate the properties of the models of curiosity. This suggests that it is possible to connect psychological findings with issues in the domain of HAI by using cognitive models.

Reminiscence therapy is mental health care based on the recollection of memories. However, the effectiveness of this method varies amongst individuals. To solve this problem, it is necessary to provide more personalized support; therefore, this study utilized a computational model of personal memory recollection based on a cognitive architecture adaptive control of thought-rational (ACT-R). An ACT-R memory model reflecting the state of users is expected to facilitate personal recollection. In this study, we proposed a method for estimating the internal states of users through repeated interactions with the memory model. The model, which contains the lifelog of the user, presents a memory item (stimulus) to the user, and receives the response of the user to the stimulus, based on which it adjusts the internal parameters of the model. Through the repetition of these processes, the parameters of the model will reflect the internal states of the user. To confirm the feasibility of the proposed method, we analyzed utterances of users when using a system that incorporates this model. The results confirmed the ability of the method to estimate the memory retrieval parameters of the model from the utterances of the user. In addition, the ability of the method to estimate changes in the mood of the user caused by using the system was confirmed. These results support the feasibility of the interactive method for estimating human internal states, which will eventually contribute to the ability to induce memory recall and emotions for our well-being.

What factors induce human empathy for virtual agents? To explore this question, we examined the relationship between humans and agents through the inverted cyberball game, wherein a participant is able to help an ostracized agent. In particular, this study attempted to distinguish factors of self-experience from situational awareness as the basis for empathy. In one scenario, participants completed a pre-task in which they had the same experience as the ostracized agent, while in another scenario, participants just observed ostracizing relations between three agents. As a result, self-experience with the pre-task did not influence the occurrence of helping behavior for the ostracized agent in the main task. Rather, the factor inducing the helping behavior was whether the participants noticed the ostracizing relations. This attentiveness was also associated with an empathetic trait, measured as the empathy quotient (EQ). From these results, we concluded that empathy is an ability to adopt others’ perspectives, even without direct or first-hand experience with others’ pain.

In everyday life, humans often try to understand and predict other’s behavior. However, the cognitive process behind this is not well understood. To clarify this, we require a model that describes the cognitive process of “behavior-reading” in detail. In this study, we proposed a card game in which behavior-reading occurs and several Adaptive Control of Thought-Rational (ACT-R) models that play the game. Besides, we conducted a simulation where models played the game against each other. The results suggest that imitation reproduces complex behavior-reading.

To improve cognitive performance with the support of computational systems such as virtual agents, it is necessary to examine changes in decision-making caused by emotions. Many studies have examined the relationship between emotions and decision-making by manipulating external factors. However, most of these studies only present the stimuli that evoke emotions and do not consider the internal characteristics of the participants. Therefore, it is necessary to consider the interactions between the internal and external factors. Based on the dual-process model of decision-making, the fast process (System 1) and the slow process (System 2) were considered as internal factors of the decision-makers. To examine the effect of each internal factor, two experiments were conducted,where participants completed a gambling task, in which multimodal external factors, such as the lightning color as a visual stimulus, the background music (BGM) as an auditory stimulus, and the facial expressions of agents as a social stimulus, were manipulated. In Experiment 1, the participants’ processes were assumed to be under System 1, as their attention was reduced in crowdsourcing experiments. In Experiment 2, the state of the participants was assumed to be under System 2 by implementing additional rewards, depending on the score and questions for rule confirmation. Consequently, the arousal level evoked by stimuli was found to have a significant impact on decision-making in Experiment 1 than in Experiment 2. The indices of decision-making that varied with stimuli were also different between the two experiments. Thus, the distinction between the two processes intermediates the effects of emotional arousal on decision-making.

To realize a harmonized relationship between human and computational systems, it is necessary to understand the nature of empathy for a virtual agent and the kinds of behaviors elicited by such mental activity. By setting a ball-tossing game in which a participant and two virtual agents were involved, we examined how ostracism experiences affect cognitive empathy and how embodiment induced by VR (Virtual Reality) influences behaviors that elicit emotional empathy. Thirty-two participants were divided into two groups by manipulating the experience of being ostracized in the preparatory task. In the main task, the participants played the game under two conditions: an equitable condition, in which the virtual agents tossed the ball evenly to the participant and the other agent, and an inequitable condition, in which one agent tossed the ball only to the participant. The data obtained in the experiment was analyzed from the viewpoint of helping behaviors to the agent who was ostracized and examined the relation of this behavior with the EQ (Empathy Quotient), an index to measure an individual's empathetic traits. As a result, the ostracized experience made the participants notice the ostracizing behaviors of the agent and fostered helping behavior toward the agent who was ostracized. In addition, the participants with a high EQ tended not to notice the ostracizing behavior in the situation without embodiment induced by VR, whereas they exhibited more helping behaviors when they experienced the task with the VR device. We consider that these findings contribute to the design of smoother interaction between humans and virtual agents.

Empathy is important for smooth interaction. In this research, we focused on EQ (Empathy Quotient), which is an index to measure individuals' empathic ability, and analyzed behavior related to EQ, particularly the effect of participants' embodiment on empathic behavior. In the experiment, participants played a ball-tossing game with two agents. The experimental task was created by adapting the Cyberball game that has been used in studies of social psychology. We conducted the experiment under two conditions: an equitable scenario where the two agents toss evenly to participants and other agents; and an inequitable scenario where only one agent tosses only to participants. Then we compared the number of tosses performed by each participants. As a result, we found that the participants with high EQ tended to exclude the agent who did not receive tosses from the other agent, but that this exclusion was alleviated by embodiment.

In the field of Human-Agent Interaction (HAI), continuation of interaction is one of the main areas of research. If the behavior of the agent is too predictable, humans stop interacting with it when they get bored. In this study, we aim to build agents that people want to keep interacting with, by employing the Adaptive Control of Thought-Rational (ACT-R) cognitive architecture. As a preliminary step, we attempt to clarify the conditions required to maintain interaction between humans and agents, by modeling the experiences of fun and boredom based on intrinsic motivation.

The eye gaze behavior of individuals changes depending on their knowledge and experience of the event occurring in their field of view. In past studies, researchers formulated a hypothesis concerning this dependency on a specific scene and then analyzed the gaze behavior of viewers observing the scene. We depart from this hypothesis-testing paradigm. In this paper, we propose a data-mining framework for extracting skilled gaze behaviors of experts while watching a video based on a comprehensive comparison of viewers in terms of the dependency of their gaze patterns on video scenes. To quantitatively analyze the changes in the gaze behavior of experts according to the events in the scene, video and eye movement sequences are classified into video scenes and gaze patterns, respectively, by using an unsupervised clustering method focusing on short-time dynamics. Then, we analyze the dependency based on the distinctiveness and occurrence frequency of gaze patterns for each video scene.

This paper proposes an approach to elderly mental care called model-based reminiscence, which utilizes cognitive modeling to guide a user's mental time travel. In this approach, a personalized cognitive model is constructed by implementing a user's lifelog (a photo library) in the ACT-R cognitive architecture. The constructed model retrieves photos based on human memory characteristics such as learning, forgetting, inhibition, and noise. These memory characteristics are regulated with parameter values corresponding to cognitive and emotional health. The authors assumed that a user's mental health could be assessed from their reactions to photo sequences retrieved by models with various parameter settings. The authors also assumed that it would be possible to motivate a user by guiding their memory recall with photo sequences generated from a healthy optimal state model. A simulation study indicates the potential of this approach presenting a variety of model behaviors corresponding cognitive / emotional states.

We built a multimodal biofeedback system integrated with low-cost sensors and applied the system to support meditation training utilizing multimodal biofeedback to the user. Our meditation support system employs Electroencephalogram, heart rate variability and eye tracking. The first two biosignals are employed to assess the mental stress during meditation, and eye tracking is equipped to detect an interval where the user engages in meditation by monitoring the open/closed states of the eyes.

For practice-based science education, the authors developed a cognitive simulator that demonstrates the human memory process and simulates the serial position effect in different experimental situations. Our cognitive simulator as a learning tool is established on the basis of the dual storage model; it visualizes the items stored in the short-term and long-term memories. The participants learn how the model works while confirming which items are rehearsed in the short-term memory, encoded into the long-term memory, or overflowed from the memory. We designed and performed practice-based psychological training through two university class sessions of the author’s cognitive science class. The results of the practice showed that participants’ data interpretation and data prediction were improved through class activities. More specifically, the participants explained the observed data using naïve concepts prior to the learning phase, but they subsequently explained them using theoretically defined concepts of the dual storage model. Further more, the participants were successfully guided to predict the experimental results more accurately by the learning activities using the cognitive simulator.

The authors have developed a web-based production system that users can use whenever and anywhere by the Internet. The authors held two cognitive science introductory classes with the system. In our class activities, participants were required to construct running cognitive models on the production system architecture that can solve pulley problems. The participants not only constructed cognitive models but also produced original problems, which were distributed to the other class members. The participants found the defects in their models while trying to solve the problems from other members and trying to improve their models. A posttest indicated that the participants who successfully constructed high performance models revealed deeper understanding of pulley systems.