User psychology remains a significant obstacle in dialogue agents. Previous studies have shown that users’ dialogue motivation decreases if they find it difficult to attribute subjective opinions to agents. This study proposes physical touch and a dialogue strategy called virtual sense acquisition statement (VSAS) to develop an interaction design for promoting opinion attribution and maintaining users’ dialogue motivation. VSAS is a dialogue strategy wherein an agent claims to have acquired a human-like sense before engaging in dialogue with a user. We examined the effect of the proposed interaction design on user dialogue motivation through a dialogue experiment (n = 20). The results showed that VSAS increased dialogue motivation in the ’Religion & Festivals’ topic category. These findings suggest that relatively simple interaction designs involving VSAS can significantly affect users’ motivation to engage in dialogue with agents.

This study proposes a dialogue strategy called “Critical Tongue Dialogue Strategy” (CTDS) for dialogue agents that respond to users experiencing anxiety or tension by delivering blunt, non-conformist remarks while maintaining a non-empathetic stance. This strategy employs balance theory principles to deliberately avoid acknowledging users’ negative emotions, instead utilizing interpersonal emotion regulation strategies. By creating emotional imbalance, it aims to trigger users’ own emotion regulation processes, thereby facilitating cognitive changes that enable them to reframe their anxieties and tensions in a more positive light. In this paper, we present the interaction modeling of this dialogue strategy and the results of psychological evaluations from a scenario-based and video-based study comparing CTDS agents with those employing empathetic dialogue strategies (n=108). Experimental results demonstrate that in terms of cognitive change assessment metrics, CTDS significantly outperformed the empathetic dialogue strategy.

This study aimed to clarify how end-users’ mind perception towards an agent (Social Connection and Self-Aware Emotions) affects the trust transference from the agent adopter (AA) to the agent. A questionnaire survey was conducted, and the results confirmed a significant interaction between trust in the AA and mind perception towards the agent concerning affective trust. This suggests that when an agent is highly perceived to have a mind, negative information about the AA strongly affects affective distrust towards the agent. However, no interaction was observed regarding cognitive trust, suggesting that the mechanisms by which humans form cognitive and affective trust in agents might differ.

A method is presented for enhancing the steering maneuvers of human drivers using peripheral visual information generated based on the vehicle’s lateral positions on the road. Human drivers control vehicles mainly using visual feedback information received from the road. However, such feedback may sometimes become degraded due to various reasons. On the other hand, recent advances in sensory technologies make it possible to detect vehicle behavior on the road quite stably. In this paper, we investigate a way to compensate such impaired feedback using artificial visual stimuli. A series of LED devices were installed in the car’s doors to display the stimuli. We designed three types of illumination patterns (flow, position, and width conditions) that react to the current lateral position of the vehicle. Simulated driving studies were conducted to evaluate the effect of each pattern on the driver’s control of the vehicle. As a result, we found that the presenting the artificial visual stimuli without explanation could significantly reduce inappropriate steering maneuvers in the degraded feedback situations.

Cybernetic avatar (CA) is an emerging technology with increasing popularity. Since CAs can be teleoperated remotely, the performance and comfort of CA operators are crucial. In this study, three different viewpoint types were evaluated: first-person view, behind-avatar view, and third-person view. In our experiment, a head-mounted display (HMD) was used to display simulated video images to the participants and track their behavior. We investigated which viewpoint type is most suitable for operators to increase usability and reduce fatigue. Participants were asked to answer a questionnaire to evaluate their satisfaction after each experimental trial. Their gaze and head movements were also analyzed. The questionnaire results suggest that the best viewing type is the behind-avatar view and the worst viewing type is the third-person view. We also found a correlation between each question and gaze movement. Our results reveal that rapid eye movements are correlated with the answer from the questionnaire. In summary, participants considered the system difficult to use and felt tired if there were many rapid eye movements in an experimental trial.

In this paper, we propose a method to improve the performance of deep gaze estimation using face-shape parameters adapted to a specific target person based on multiple observations. Our gaze estimation network contains a predefined computation module that calculates gaze directions using known geometric relationships among head poses, eye-ball positions, and gaze directions. Updated face-shape parameters contribute to improving the performance of the process. In addition, the computation module enables a network to acquire the ability to induce hidden parameters such as eyeball position and eyeball radius from observed information through a training process. Experimental results reveal improvement in gaze estimation accuracy by introducing a sequential update process for face-shape parameters and a predefined computation module.

This research aims to develop a method to assist iterative visual search tasks, and it focuses on visual cognition to achieve effective assistance. As a first step to this goal, we analyzed the participants’ gaze behaviors when they visually recognized a target in a VR environment. In the experiment, the effect of visual cognition difficulty (VCD) is considered. Analysis results show that the participants could visually recognize lower-VCD targets at an earlier timing. This suggests that VCD-based guidance may improve task performance.

This paper addresses the differences between conventional optical mirrors and electrical mirrors consisting of a camera and a flat display panel. Recently, vehicle mirrors (side-view and rear-view mirrors) are gradually being replaced with electrical mirrors (i.e., display devices). The characteristics of electrical mirrors are different from those of conventional mirrors in many aspects. These differences can cause an uncomfortable feeling in drivers. In this paper, we focus on the half occlusion appearing in the peripheral areas of a display and investigate the effects of half-occluded regions on whether humans recognize a viewing device as a mirror. To evaluate this effect, we conducted experiments based on pairwise comparisons using a head-mounted display (HMD) to present virtual mirror objects having different characteristics in terms of half occlusion as well as binocular and motion parallaxes. Experimental results suggest that half occlusion is certainly related to the human recognition of a mirror. Furthermore, pseudo half-occlusions introduced by barriers in front of a 2D display can reinforce this recognition.

Automated driving reduces the burden on the driver, however also makes it difficult for the driver to understand the current situation and predict the future movement of the vehicle. When the acceleration due to automated driving occurs without future prediction, the driver's anxiety and discomfort are increased compared to the case in manual driving. To facilitate the prediction of the future behavior of the vehicle by the driver, this paper aims to design and evaluate a haptic interface that actuates the vehicle seat. Our system displays to the driver the movement of the vehicle a few seconds in the future, which allows the driver to make predictions and preparations. Using a driving simulator, we compared the conditions where the movement of the car was displayed in advance for the length of different time. The subjective evaluation of the driver showed that the predictability of the behavior of the vehicle were significantly increased compared to the case without display. The experiment also showed that comfortable feeling significantly decreased if the preceding display is too early.

In this paper, we propose a simple method for refining pretrained word embeddings using layer-wise relevance propagation.Given a target semantic representation one would like word vectors to reflect, our method first trains the mapping between the original word vectors and the target representation using a neural network.Estimated target values are then propagated backward toward word vectors, and a relevance score is computed for each dimension of word vectors.Finally, the relevance score vectors are used to refine the original word vectors so that they are projected into the subspace that reflects the information relevant to the target representation.The evaluation experiment using binary classification of word pairs demonstrates that the refined vectors by our method achieve the higher performance than the original vectors.

This paper investigates visual attention control using the presentation of directional flow stimulus to peripheral vision. Peripheral vision is known to have a superior motion-perception capability. Since central vision is usually used for a primary visual task, it would be quite useful if we could control one's attention by providing assistive information through peripheral motion cues without interfering with the primary task. To evaluate the effectiveness of the proposed method, we conducted experiments on rapid target recognition and visual search tasks under peripheral stimulation conditions. As a result, in the target recognition task, we confirmed that the position with the highest recognition score corresponds to the direction of the presented flow stimuli. In a visual search task, response time decreases when the target position and flow direction match. Furthermore, such matching allows the subjects to more quickly learn how to use the presented information in their search task. Subjective evaluation by questionnaire also demonstrates the intuitiveness and helpfulness of the proposed method. These results support the effectiveness of attention control and visual search assistance using the presentation of directional flow stimuli to peripheral vision.

In this paper, we experimentally examine the relationship between visual cognition difficulty and target-tracking eye movements, which recorded during moving target cognition. Generally, such eye movements are observed when humans perceive a moving object and they vary widely due to many factors, such as target shape, backgrounds, illumination conditions, and so on. Several systems have been proposed for estimating human cognition based on gaze movements. However, since most of them employ simple thresholding techniques to classify the states of cognition, their classification performance remain insufficient. This research clarifies the relationship between multiple visual conditions and target-tracking eye movements to enhance the classification performance. We found that we can address a variety of target and background factors from the perspective of visual cognition difficulty for the targets. Observed gaze movement properties and the difficulty of target visual cognition have a linear relationship, suggesting the possibility of more precise estimation of human's target cognition based on them.

Abstract This study examines the ability of a semantic space model to represent the meaning of noun compounds such as ‘information gathering’ or ‘heart disease.’ For a semantic space model to compute the meaning and the attributional similarity (or semantic relatedness) for unfamiliar noun compounds that do not occur in a corpus, the vector for a noun compound must be computed from the vectors of its constituent words using vector composition algorithms. Six composition algorithms (i.e., centroid, multiplication, circular convolution, predication, comparison, and dilation) are compared in terms of the quality of the computation of the attributional similarity for English and Japanese noun compounds. To evaluate the performance of the computation of the similarity, this study uses three tasks (i.e., related word ranking, similarity correlation, and semantic classification), and two types of semantic spaces (i.e., latent semantic analysis-based and positive pointwise mutual information-based spaces). The result of these tasks is that the dilation algorithm is generally most effective in computing the similarity of noun compounds, while the multiplication algorithm is best suited specifically for the positive pointwise mutual information-based space. In addition, the comparison algorithm works better for unfamiliar noun compounds that do not occur in the corpus. These findings indicate that in general a semantic space model, and in particular the dilation, multiplication, and comparison algorithms have sufficient ability to compute the attributional similarity for noun compounds.

The ink drop spread (IDS) method is a modeling technique based on the idea of soft computing. This method divides a multi-input-single-output (MISO) target system into multiple single-input-single-output (SISO) systems, and models each SISO system by plotting the input/output data. The IDS method combines the modeling results of SISO systems to model the target. It is important for the IDS method to decide appropriate partitions of the target system in order to accurately model the target. Existing partitioning methods divide each input domain independently of the other inputs, and thus generate unnecessary SISO systems. In this article, we propose a new partitioning method for the IDS method, which divides the input domains by considering the relationship between inputs. We also show that our method can achieve better performance with less partitions than existing methods.

A "private information detector (PID)" is described that helps control the circulation of a user's private information through a social network. It checks texts to be posted by the user on a social network and detects potential revelations of private information so that it warns the user or modifies the texts automatically. It can cope with a wide variety of expressions that might be revealing by using public information accessible through the Internet rather than a large knowledge base. Evaluation using about 7000 blog sentences showed that it has reasonably good performance in terms of true and false detection rates.

We propose a multi-camera-based gaze tracking system that provides a wide observation area. In our system, multiple camera observations are used to expand the detection area by employing mosaic observations. Each facial feature and eye region image can be observed by different cameras, and in contrast to stereo-based systems, no shared observations are required. This feature relaxes the geometrical constraints in terms of head orientation and camera viewpoints and realizes wide availability of gaze tracking with a small number of cameras. In experiments, we confirmed that our implemented system can track head rotation of 120° with two cameras. The gaze estimation accuracy is 5.4° horizontally and 9.7° vertically.

This study examines the ability of nonnegative matrix factorization (NMF) as a method for constructing semantic spaces, in which the meaning of each word is represented by a high-dimensional vector. The performance of two tests (i.e., a multiple-choice synonym test and a word association test) is compared between NMF and latent semantic analysis (LSA), which is the most popular method for constructing semantic spaces. As a result, it was found that NMF did not outperform LSA in either test. This finding indicates that NMF is less effective in acquiring word meanings than expected in the literature; in other words, the finding provides evidence for the ability of LSA to represent semantic meanings. Some properties of NMF were also revealed with reference to its ability to represent word meanings; the random initialization was superior to the SVD-based initialization, and the Euclidean distance is more appropriate for the objective function of NMF than the KL-divergence. In addition, it was shown that the inner product was a more appropriate method for measuring the syntagmatic similarity in a semantic space model, while the cosine was a better method for computing the paradigmatic similarity.

We propose a real-time gaze estimation method based on facial-feature tracking using a single video camera that does not require any special user action for calibration. Many gaze estimation methods have been already proposed; however, most conventional gaze tracking algorithms can only be applied to experimental environments due to their complex calibration procedures and lacking of usability. In this paper, we propose a gaze estimation method that can apply to daily-life situations. Gaze directions are determined as 3D vectors connecting both the eyeball and the iris centers. Since the eyeball center and radius cannot be directly observed from images, the geometrical relationship between the eyeball centers and the facial features and eyeball radius (face/eye model) are calculated in advance. Then, the 2D positions of the eyeball centers can be determined by tracking the facial features. While conventional methods require instructing users to perform such special actions as looking at several reference points in the calibration process, the proposed method does not require such special calibration action of users and can be realized by combining 3D eye-model-based gaze estimation and circle-based algorithms for eye-model calibration. Experimental results show that the gaze estimation accuracy of the proposed method is 5° horizontally and 7° vertically. With our proposed method, various application such as gaze-communication robots, gaze-based interactive signboards, etc. that require gaze information in daily-life situations are possible.

In this paper, we propose a guide system for daily life in semipublic spaces by adopting a gaze-communicative stuffed-toy robot and a gaze-interactive display board. The system provides naturally anthropomorphic guidance through a) gaze-communicative behaviors of the stuffed-toy robot (ldquojoint attentionrdquo and ldquoeye-contact reactionsrdquo) that virtually express its internal mind, b) voice guidance, and c) projection on the board corresponding to the userpsilas gaze orientation. The userpsilas gaze is estimated by our remote gaze-tracking method. The results from both subjective/objective evaluations and demonstration experiments in a semipublic space show i) the holistic operation of the system and ii) the inherent effectiveness of the gaze-communicative guide.

This paper proposes a gaze-communicative stuffed-toy robot system with joint attention and eye-contact reactions based on ambient gaze-tracking. For free and natural interaction, we adopted our remote gaze-tracking method. Corresponding to the user's gaze, the gaze-reactive stuffed-toy robot is designed to gradually establish 1) joint attention using the direction of the robot's head and 2) eye-contact reactions from several sets of motion. From both subjective evaluations and observations of the user's gaze in the demonstration experiments, we found that i) joint attention draws the user's interest along with the user-guessed interest of the robot, ii) "eye contact" brings the user a favorable feeling for the robot, and iii) this feeling is enhanced when "eye contact" is used in combination with "joint attention." These results support the approach of our embodied gaze-communication model.

In this paper, we describe our methods of detecting human behavior in order to monitor and assist daily human tasks. To assist people in performing activities in daily life, we must be able to understand the system user's situation/state in the current task: what information is useful for the user now? Our posture-detection system using IR cameras and invisible IR pattern projectors detects the user's state as his/her 3D appearance (posture). In the proposed system, human behavior is modeled as a distribution of 3D appearances using kernel density functions, and the results of this behavior detection are used to determine the instructions to be given to the user. We show a sample implementation of the system for a toilet task with voice- and CG-animation-based instructions to users. By using this method, we can achieve effective task assistance and reveal limited visual representation of users, taking into account the human need for personal privacy. Finally, we show experimental results of using the proposed method for human tracking and behavior monitoring

This paper proposes a cognitive model of how poetic effects are achieved by a work of literature, especially by individual figurative expressions such as metaphor and irony. According to the proposed model, poetic effects are evoked during interpretation of a figurative expression, if great processing effort caused by an incongruity involved in the figurative expression is rewarded by a rich interpretation which happens too fast to adjust to what is happening. This paper provides a comprehensive explanation of the poetic effects for metaphor and irony based on the proposed model, together with some related discussions. Finally, it suggests a possibility that the proposed model can be extended to the relevant phenomena such as humor.

We propose an adaptive human tracking system with non-synchronous multiple observations. Our system consists of three types of processes: discovering node for detecting newly appeared person; tracking node for tracking each target person; and observation node for processing one viewpoint (camera) images. We have multiple observation nodes and each node works independently. The tracking node integrates the observed information based on reliability evaluation. Both the observation conditions and human motion states are considered in the evaluation. Matching between tracking models and observed image features are performed in each of the observation node based on the position, size and color similarities of each 2D image. Due to the non-synchronous property, this system is highly scalable for increasing the detection area and number of observing nodes. Experimental results for some indoor scenes are also described.

We propose a method of tracking 3D position, posture, and shapes of human hands from multiple-viewpoint images. Self-occlusion and hand-hand occlusion are serious problems in the vision-based hand tracking. Our system employs multiple-viewpoint and viewpoint selection mechanism to reduce these problems. Each hand position is tracked with a Kalman filler and the motion vectors are updated with image features in selected images that do not include hand-hand occlusion. 3D hand postures are estimated with a small number of reliable image features. These features are extracted based on distance transformation, and they are robust against changes in hand shape and self-occlusion. Finally, a "best view" image is selected for each hand for shape recognition. The shape recognition process is based on a Fourier descriptor. Our system can be used as a user interface device an a virtual environment, replacing glove-type devices and overcoming most of the disadvantages of contact-type devices.

We propose a novel method of extracting a moving object region from each frame in a series of images regardless of complex, changing background using statistical knowledge about the target. In vision systems for 'real worlds' like a human motion tracer, a priori knowledge about the target and environment is often limited (e.g., only the approximate size of the target is known) and is insufficient for extracting the target motion directly. In our approach, information about both target object and environment is extracted with a small amount of given knowledge about the target object. Pixel value (color, intensity, etc.) distributions for both the target object and background region are adaptively estimated from the input image sequence based on the knowledge. Then, the probability of each pixel being associated with the target object is calculated. The target motion can be extracted from the calculated stochastic image. We confirmed the stability of this approach through experiments.

We propose a multiple-view-based tracking algorithm for multiple-human motions. In vision-based human tracking, self-occlusions and human-human occlusions are a part of the more significant problems. Employing multiple viewpoints and a viewpoint selection mechanism, however can reduce these problems. In our system, human positions are tracked with a sequence of multiple-viewpoint images. This tracking is based on the Kalman filtering approach. The estimation results are utilized to select proper viewpoints in other sub-tasks (rotation angle detection and body-side detection). Each sub-task has a different criterion for selecting viewpoints. We also describe the criterions for accomplishing individual sub-tasks and relationships between sub-tasks. We have already built an experimental system based on a small number of reliable image features. We confirm the stability of our algorithm through simulations. We also performed fundamental examinations on the experimental system.

Two methods to extract a moving target region from a series of images are presented. Pixel value distributions for both the target object and background region are estimated for each pixel with roughly extracted moving regions. Using the distributions, stable target extraction is performed. In the first method, the distributions are approximated with Gaussian distribution functions and the probability of a pixel being associated with the target object is calculated. In the second method, a Markov random field model is applied to perform region segmentation on regularized input images using the estimated pixel value distributions. The texture parameters for the target object region can be calculated from the estimated pixel value distributions. Experimental results obtained by these two methods using hand motion images are presented.

We describe a method to detect hand position, posture and finger bendings using multiple camera images. Stable detection can be achieved by using skeleton images, and this is confirmed through experiments. This system can be used as a user interface device in a virtual environment, replacing glove-type devices and overcoming most of the disadvantages of contact-type devices. Future work includes verification of the method's availability as a gesture-based man-machine interface system.