Although it is important to understand learners’ emotional situations in teaching and learning processes, learners sometimes do not express negative emotions intentionally. This study has termed this intentional non-expression of negative emotions “emotional concealment,” and examined its estimation from learners’ biometric information. In particular, we focused on learners’ facial features, and attempted to clarify their relationship with emotional concealment. Using these results, we attempted to quantify emotional concealment from the time-series data of biometric information and detect it as an outlier. Additionally, we attempted to develop a feedback generation system to inform teachers about emotional concealment.

Cognitive architecture plays essential role in Intelligent Tutoring System (ITS). For effective learning support, dynamic model subserving the function of the prediction of learners’ states should be incorporated. In this paper, we proposed a neural network-based architecture in progress that can incorporate the dynamic model to generate the prediction of leaners’ perceptual states when categorical ambiguous stimulus and the instruction of category were given.

In this study, we evaluated the mental stress due to corner entry speed while riding a mobility scooter by measuring skin conductance. A mobility scooter is a four-wheel electric wheelchair with a steering bar-handle. These scooters, known as “senior cars” in Japan, are generally sold for use by elderly people. Some of these scooters have a system that automatically reduces their speed to approximately 70% of the maximum depending on the steering angle at corners. However, because of mismatch with the intention of the driver, the preset deceleration rate may be uncomfortable for the driver. With the objective of providing appropriate assistance without any discomfort in such circumstances, we evaluated the mental stress of the driver. Skin conductance (SC) is one index for measuring mental sweating related to sympathetic nerve activity. First, we modified a commercially available mobility scooter for use as an experimental scooter such that it can be controlled using an external personal computer. Then, we measured SC in a task that involved turning right into a corner at a constant speed using the experimental scooter, and evaluated the mental stress due to the corner entry speed. A positive correlation between the amplitude of SC and entry speed was obtained. Further, the results of a questionnaire used to evaluate the ride discomfort after each experimental task indicated that “No Fear” and “Safety” decreased as the speed increased. As these results are consistent with the measurement results of the SC amplitude, we concluded that fear due to corner entry speed can be evaluated using SC.

Currently, visually impaired people find it difficult to use mobility support facilities because they depend on properties of the ambient environment, such as ambient sound and snow cover. Furthermore, people who became visually impaired in later life also find it difficult to use these facilities. Therefore, in this study, we aimed to develop a walking guidance system, using three-dimensional (3D) sound, that is not influenced by the surrounding environment and is easy to use. We verified the characteristics of the guidance sound produced by 3D sound. This system was implemented with a head-mounted display that is capable of producing 3D sound. In a visual restriction environment, blindfolded participants were asked to indicate toward a real or 3D guiding sound with two conditions: with head fixed or free, and using a tweet-like or water sound. We also collected subjective evaluations of the sounds. As a result, we found that a stationary sound including frequency components in a wide band, like a water sound, was able to be localized with a higher accuracy than a tweet-like sound, and was suitable as a guidance sound, according to subjective evaluations.

Fragrance influences the psychophysiological state of a person; the volume of fingertip plethysmography is consequently affected by the aspiration of aroma. Further, it may be uncomfortable depending on the amount of odor that you inhales. In this study, to observe changes in emotions during aroma aspiration, the flow of scent was continuously directed into the nose of participants quantitatively and directly. Subsequently, the stimulation threshold during aroma inhalation and change in emotions were evaluated through a questionnaire and fingertip plethysmography. We developed a fragrance generator to ensure quantitative aroma flow and participants were asked to subjectively evaluate aroma intensity and the level of comfort in real time. The results confirmed the chaotic nature of the time series data of fingertip plethysmographs. Moreover, it was confirmed that the Largest Lyapunov Exponent (LLE), which provides an indication of the amount of fluctuation, corresponds to the subjective evaluation of comfort/discomfort. The value of the index decreases during pleasure and increases during discomfort. In other words, it was confirmed that the pulse wave fluctuated when the participants were experiencing discomfort.

In the recent years, research on intelligent educational systems has attracted significant interest in exploring data from educational settings to understand learners' behavior and mental states. Hence, the means to deduce the mental state of a learner has been obtained from a variety of physiological indices. The current study aims to understand how a learner answers a question with confidence. By measuring the brain activity and eye movement by using a set of multiple-choice questions, this study analyzes the relationships between the confidence of answering the question, brain activation and eye movement at the time of answering. When there was a high confidence of answering, oxyHb tended to increase near the prefrontal cortex frontal pole and sight line movement to reread question text tended to decrease. The results show that this technique may be able to capture the mental state of the learner by measuring a combination of the change of oxyHb and sight line movement.

We developed a walking assistance apparatus which can be used for gait training of neuro-rehabilitation for patients and for the promotion of exercise for the elderly. This apparatus assists only the ankle joints of the equipped person according to the posture of him/her while not only walking, but also getting up, sitting, and standing. By using this apparatus, the dorsiflexion and plantarflexion of the ankle joint of the equipped person can increase while walking. This apparatus can improve of the equipped person's gait.

Many studies have been conducted during the last two decades examining learner reactions within e-learning environments. In an effort to assist learners in their scholastic activities, these studies have attempted to understand learner mental states by analyzing participants’ facial images, eye movements, and other physiological indices and data. To add to this growing body of research, we have been developing IMS (Intelligent Mentoring System) which performs automatic mentoring by using an ITS (Intelligent Tutoring System) to scaffold learning activities and an ontology to provide a specification of learner’s models. To identify learner mental states, the ontology operates based on theoretical and data-driven knowledge of emotions. In this study, we use statistical models to examine constructs of emotions evaluated in previous psychological studies, and then produce a construct of academic boredom.

Recently, the research in intelligent educational systems has much interest in exploring data from academic settings to understand learners behavior and mental states. We have been developing IMS (Intelligent Mentoring System) which performs automatic mentoring by using an ITS (Intelligent Tutoring System) to scaffold learning activities and an ontology to provide a specification of learner ’s models. To provide learner ’s models in mentoring, the ontology describes qualities and quality values on awareness which are conceptualization of subjective evaluation. To specify relationships among qualities on awareness in academic settings, this study organized academic emotions in the psychological research and proposed their ontological descriptions.

Recently, the research area of intelligent educational systems has increasingly approached Educational Data Mining, which addressed a variety of learner aspects such as mental states. Although data of eye movements is promising in realizing mental states of learners, we have to empirically understand features of eye movements in advance due to difficulty in handling them. Toward the final goal of automatic estimation of learner mental states based on eyes, this study experimentally described changes of eye movements by confidence of correct answers. In our experiment, participants were asked to answer multiple-choice problems and respond to questionnaires about the problems, with the results indicating that transitions of eyes in initial processes where all choices were scanned differed depending on the levels of confidence.

Recently, the research in intelligent educational systems has much interest in exploring data from educational settings to understand learners behavior and mental states. To further understand learners based on such the data and sophisticate supportive interventions by educational systems, knowledge of relationships between learners' behavior and mental states must be shared. To propose a framework to share the knowledge, this study attempted ontological descriptions for learners’ eye-movement and mental states in solving computer-based multiple-choice problems. The current study forms a technical basis for development of an IMS (Intelligent Mentoring System) in which an automatic mentoring function is implemented with ITS (Intelligent Tutoring System).