Educational robots prove effective in children’s learning, but designing robots that support parent-child dialogue during home learning remains unclear. This study proposes a semi-autonomous para-operated robot that parents partially control while engaging with their child. We hypothesized that unlike autonomous robots which may reduce parental engagement [1], this approach would increase parental involvement through active operation. We conducted an experiment with parent-child pairs comparing no robot, autonomous robot, and the proposed semi-autonomous robot conditions. Results showed the proposed robot significantly increased parental utterances without increasing parental burden, demonstrating its effectiveness in facilitating parent-child home learning dialogue.

In personality trait analysis, self-evaluation through questionnaire methods is widely used due to ease of implementation and analysis. However, respondents can easily provide false evaluation scores based on their intentions. This study proposes a method to obtain truthful evaluation scores from dialogue records through dialogue simulation with autonomous agents for questionnaire items. Since dialogue requires immediate responses and detailed explanations, we hypothesized this method would be less prone to deception compared to traditional questionnaire methods. We developed a comprehensive system using Large Language Models that handles everything from setting dialogue situations based on questionnaire items to conducting dialogues with autonomous agents, and automatically evaluating dialogue records. In preliminary experiments, we found that evaluation scores from the proposed method when participants were prompted to lie were close to truthful questionnaire responses when participants were prompted to answer truthfully. These findings suggest the effectiveness of psychological measurement through dialogue with autonomous agents.

Agents that migrate across different devices can provide seamless support for users in various contexts. However, little is known about how users determine whether a migrating agent remains the same individual across different devices. To explore the key factors influencing users’ perception of agent continuity before and after migration, we conducted two experiments. In Experiment 1, we examined how users’ perception is influenced when two visually identical agents differ in any of the three factors: voice, memory, or speaking style. Experiment 2 tested whether maintaining the factors identified as critical in Experiment 1 would enable users to perceive the agent as the same individual after it migrated from a robot to headphones. Results suggest that while changes in the factors can disrupt perceived continuity, preserving them enables consistent identity perception before and after migration, even when the agent’s appearance changes.

Co-located operated robots, where an operator, a robot, and a user interact in the same physical space, are an emerging paradigm in human-robot interaction. This para-operation approach aims to reduce operator workload and enhance communication. A key challenge is balancing robot autonomy with operator intent; fully manual systems lack flexibility, while fully autonomous ones risk unintended actions. To address this, we propose a semi-autonomous system that uses an augmented reality (AR) interface for intuitive control. We developed a prototype where an operator’s AR glasses display contextual action choices for the robot. A preliminary user study was conducted in a simulated retail scenario. Results from interviews showed that the desired dialogue strategy depends on the operator’s proficiency. Novices preferred robot-led interaction for guidance, while experienced users favored operator-led support for nuanced control. These findings highlight the need for adaptive dialogue strategies in AR-based para-operation, offering initial insights for this collaborative framework.

An ITACO agent is a conversational agent that migrates across multiple devices to provide user support through the most suitable embodiment while maintaining a continuous relationship with the user. However, because the agent must operate on diverse devices with unique configurations and behaviors, scaling the system significantly increases implementation complexity and development effort. To address this, we propose a platform that serves as a foundational framework to streamline ITACO agent development. Our system simplifies the process of integrating new devices and defining their behaviors, enabling scalable, efficient development of migrating agents. In this paper, we present the design and implementation of this platform and report on a preliminary user study conducted to evaluate its viability. The primary contribution of our work is the core design of a foundational system that facilitates the future development of ITACO agents.

Teleoperated customer service robots have attracted attention to improve customer service efficiency. However, operators experience boredom during long-duration operation due to monotony and idle time, leading to decreased task motivation. This study proposes and evaluates a method to reduce operator boredom through dialogue with the robot to be operated. Field experiments demonstrated that operator-robot dialogue significantly reduced boredom and contributed to maintaining task engagement during long-duration operation.

This paper presents our work on a task of automatic decomposition of text editing examples into primitive edit operations. Toward a detailed analysis of the behavior of text editing systems, identification of fine-grained edit operations performed by the systems is essential. Given a pair of source and edited sentences, the goal of our task is to generate a non-redundant sequence of primitive edit operations, i.e., the semantically minimal edit operations preserving grammaticality, that iteratively converts the source sentence to the edited sentence. First, we formalize this task, explaining its significant features and specifying the constraints that primitive edit operations should satisfy. Then, we propose a method to automate this task, which consists of two steps: generation of an edit operation lattice and selection of an optimal path. To obtain a wide range of edit operation candidates in the first step, we combine a phrase aligner and a large language model. Experimental results show that our method perfectly decomposes 44% and 64% of editing examples in the text simplification and machine translation post-editing datasets, respectively. Detailed analyses also provide insights into the difficulties of this task, suggesting directions for improvement.

Interactive systems that enable people to easily access mental health care in their daily lives are being studied. Self-instruction training, which is a kind of cognitive-behavioral therapy, is expected to be effective for general-purpose mental health care, but the difficulty for users to create self-instructional texts by themselves has hindered its widespread use. Furthermore, it is not sufficient to simply use a computer-assisted method for the training in which the computer automatically generates the self-instructional texts. This is because previous studies have shown that users do not feel self-instructional texts that are not created by themselves are their thoughts, and consequently cannot internalize the content of the texts, reducing the effectiveness of the training. In this study, we focused on the interface design that facilitates users to internalize automatically generated self-instructional texts, toward the realization of a self-instructional training support system. We proposed an interface that shows the real-time captured video of a user's face like a mirror and superimposes a speech bubble containing a self-instructional text to be connected to the user's face and confirmed the interface facilitates the user's internalization of the text through online experiments. The contribution of this paper is to provide a novel interface design for computer-assisted self-instructional training.

A Manhattan world lying along cuboid buildings is useful for camera angle estimation. However, accurate and robust angle estimation from fisheye images in the Manhattan world has remained an open challenge because general scene images tend to lack constraints such as lines, arcs, and vanishing points. To achieve higher accuracy and robustness, we propose a learning-based calibration method that uses heatmap regression, which is similar to pose estimation using keypoints, to detect the directions of labeled image coordinates. Simultaneously, our two estimators recover the rotation and remove fisheye distortion by remapping from a general scene image. Without considering vanishing-point constraints, we find that additional points for learning-based methods can be defined. To compensate for the lack of vanishing points in images, we introduce auxiliary diagonal points that have the optimal 3D arrangement of spatial uniformity. Extensive experiments demonstrated that our method outperforms conventional methods on large-scale datasets and with off-the-shelf cameras.

This study investigates the effects of both operator and avatar gender on the perceived ease of communication and creepiness of the avatars. Utilizing a mixed-methods 2 by 2 approach (male operator, female operator; male avatar, female avatar), we conducted an experiment involving 148 participants. The results indicate significant variations in perceived avatar impressions influenced by the gender alignment between the operator and the avatar. Specifically, female-operator-female-avatar group rated as the easiest to talk to, most interesting and least in the creepiness metric. These results suggest that for low-friction conversational encounters using female-operator-female-avatars might be most suitable.

Teleoperated social robots have been widely studied and employed, yet their operation interfaces provide limited information to the operator, causing difficulties in grasping the dialogue situation compared to face-to-face interactions. Particularly with conventional interfaces, operators face challenges with vocal adaptation due to the inability to assess the distance between the robot and the interlocutor. Consequently, operators struggle to determine the appropriate volume level for clear communication, leading to insecurity about their vocalization’s effectiveness. In this study, we propose a novel approach to address this issue: a voice volume gauge that enables operators to adjust their voice volume by displaying the difference between their current volume and the estimated optimal volume on the interface. We implemented the gauge and conducted a subjective evaluation experiment, which demonstrated no usability concerns for operators and a reduction in their insecurity levels. Furthermore, preliminary objective evaluation results based on speech analysis suggest that utilizing the gauge may bring operators’ vocalizations closer to those in face-to-face dialogues. The contribution of this paper lies in providing a simple interface design method to aid in solving the vocal adaptation problem for teleoperated robot applications.

Various customer service robots’ teleoperation interfaces (I/Fs) have been developed for human-robot collaboration. However, a previous study has indicated a lack of operator enjoyment. This study aims to design an I/F that elicits operator enjoyment and identifies the factors that contribute to this enjoyment. We developed two I/Fs: a high degree of freedom I/F and a gradual flexibility degree of freedom I/F based on gamification to elicit operator enjoyment. Using these I/Fs, we conducted a field study in a real shopping mall to investigate the operators’ experiences. As a result, the operators in this experiment greatly enjoyed using our I/Fs. The results showed that our I/Fs could elicit operator enjoyment with the following three I/F factors suggested as potentially important design guidelines: ease of use through anonymity, moderate restriction of freedom, and sharing experiences with a group of operators.

Japanese sentence-ending predicates intricately combine content words and functional elements, such as aspect, modality, and honorifics; this can often hinder the understanding of language learners and children.Conventional lexical simplification methods, which replace difficult target words with simpler synonyms acquired from lexical resources in a word-by-word manner, are not always suitable for the simplification of such Japanese predicates.Given this situation, we propose a BERT-based simplification method, the core feature of which is the high ability to substitute the whole predicates with simple ones while maintaining their core meanings in the context by utilizing pre-trained masked language models.Experimental results showed that our proposed methods consistently outperformed the conventional thesaurus-based method by a wide margin.Furthermore, we investigated in detail the effectiveness of the average token embedding and dropout, and the remaining errors of our BERT-based methods.

Ideally, the users of spoken dialogue systems should be able to speak at their own tempo.The systems thus need to correctly interpret utterances from various users, even when these utterances contain disfluency.In response to this issue, we propose an approach based on a posteriori restoration for incorrectly segmented utterances.A crucial part of this approach is to classify whether restoration is required or not.We improve the accuracy by adapting the classifier to each user.We focus on the dialogue tempo of each user, which can be obtained during dialogues, and determine the correlation between each user's tempo and the appropriate thresholds for the classification.A linear regression function used to convert the tempos into thresholds is also derived.Experimental results showed that the proposed user adaptation for two classifiers, thresholding and decision tree, improved the classification accuracies by 3.0% and 7.4%, respectively, in ten-fold cross validation.

Appropriate turn-taking is important in spoken dialogue sys-tems as well as generating correct responses. We have devel-oped a method that performs a posteriori restoration of incor-rectly segmented utterances caused by erroneous voice activity detection (VAD), which result in automatic speech recognition (ASR) errors and inappropriate turn-taking. A crucial part of the method is to classify whether the restoration is required or not. We cast it as a binary classification problem detecting originally single utterances from pairs of utterance fragments. Various features are used representing timing, prosody, and ASR result information to improve its accuracy. Furthermore, two kinds of feature selection are performed to obtain effective and domain-independent features. The experimental results showed that the proposed method outperformed a baseline with manually-selected features by 4.8 % and 3.9 % in cross-domain evalua-tions with two domains. More detailed analysis revealed that the dominant and domain-independent features were utterance intervals and results from the Gaussian mixture model (GMM). Index Terms: spoken dialogue system, VAD error, turn taking, a posteriori restoration

The growing need for text mining systems, such as opinion mining, requires a deep semantic understanding of the target language. In order to accomplish this, extracting the semantic information of functional expressions plays a crucial role, because functional expressions such aswould like toandcan’tare key expressions to detecting customers’ needs and wants. However, in Japanese, functional expressions appear in the form of suffixes, and two different types of functional expressions are merged into one predicate: one influences the factual meaning of the predicate while the other is merely used for discourse purposes. This triggers an increase in surface forms, which hinders information extraction systems. In this article, we present a novel normalization technique that paraphrases complex functional expressions into simplified forms that retain only the crucial meaning of the predicate. We construct paraphrasing rules based on linguistic theories in syntax and semantics. The results of experiments indicate that our system achieves a high accuracy of 79.7%, while it reduces the differences in functional expressions by up to 66.7%. The results also show an improvement in the performance of predicate extraction, providing encouraging evidence of the usability of paraphrasing as a means of normalizing different language expressions.

This paper describes a system that automatically generates shop-specific listing ads by reusing textual data promoting each shop. As the textual data available for this research are primarily created for use on a restaurant portal site on the Web, we applied three methods for making it usable for the descriptive texts of ads. The only manual task is creating a couple of domain-specific patterns. Subjective evaluation showed that our system can generate ads with sufficiently high precision and coverage. A one-month experiment using Overture Sponsored Search showed that a number of the automatically generated ads had higher CTRs than the template-based baseline ads. This indicates that automatically generated ads can promote shops more effectively than template-based ads.

We have developed a web-based transliteration system of person names; from a person name written in English (Latin script), the system produces its Japanese (Katakana) transliteration extracted from the Web. Experiments have shown that the performance is sufficiently high: for 89.4% of English person names, the system produced one or more acceptable Japanese transliterations; 98.5% of system’s outputs were acceptable transliterations. This system was used for automatic compilation of an English-Japanese person-name lexicon with 406K entries.

Automatic compilation of lexicon is a dream of lexicon compilers as well as lexicon users. This paper proposes a system that crawls English-Japanese person-name transliterations from the Web, which works a back-end collector for automatic compilation of bilingual person-name lexicon. Our crawler collected 561K transliterations in five months. From them, an English-Japanese person-name lexicon with 406K entries has been compiled by an automatic post processing. This lexicon is much larger than other similar resources including English-Japanese lexicon of HeiNER obtained from Wikipedia.

This paper describes a method of translating a predicate-argument structure of a verb into that of an equivalent verb, which is a core component of the dictionary-based paraphrasing. Our method grasps several usages of a headword and those of the def-heads as a form of their case frames and aligns those case frames, which means the acquisition of word sense disambiguation rules and the detection of the appropriate equivalent and case marker transformation.

This paper describes a system for finding phrasal translation correspondences from parallel parsed corpus that are collections paired English and Japanese sentences. First, the system finds phrasal correspondences by Japanese-English translation dictionary consultation. Then, the system finds correspondences in remaining phrases by using sentences dependency structures and the balance of all correspondences. The method is based on an assumption that in parallel corpus most fragments in a source sentence have corresponding fragments in a target sentence.