Designing policies that are both efficient and acceptable for conversational service robots in open and diverse environments is non-trivial. Unlike fixed, hand-tuned parameters, online learning can adapt to non-stationary conditions. In this paper, we study how to adapt a social robot’s speech policy in the wild. During a 12-day in-situ deployment with over 1,400 public encounters, we cast online policy optimization as a multi-armed bandit problem and use Thompson sampling to select among six actions defined by speech rate (slow/normal/fast) and verbosity (concise/detailed). We compare three complementary binary rewards–Ru (user rating), Rc (conversation closure), and Rt (≥2 turns)–and show that each induces distinct arm distributions and interaction behaviors. We complement the online results with offline evaluations that analyze contextual factors (e.g., crowd level, group size) using video-annotated data. Taken together, we distill ready-to-use design lessons for deploying online optimization of speech policies in real public HRI settings.

We report a mixed-methods field experiment of a conversational service robot deployed under everyday staffing discretion in a live bedding store. Over 12 days we alternated three conditions--Baseline (no robot), Robot-only, and Robot+Fixture--and video-annotated the service funnel from passersby to purchase. An explanatory sequential design then used six post-experiment staff interviews to interpret the quantitative patterns.

Voice is an essential modality for human-robot interaction (HRI). The way a robot sounds plays a central role in shaping how humans perceive and engage with it, influencing factors such as intelligibility, understandability, and likability. Although prior work has examined voice design, most studies occur in controlled labs, leaving uncertainty about how results translate to real-world settings. To address this gap, we conducted two naturalistic deployment studies with a guidance robot in a shopping mall: (1) in-depth interviews with six participants, and (2) an eight-day field deployment using a 3×3 design varying speech rate and volume, yielding 725 survey responses. Our results show how real-world context shapes voice perception and inform adaptive, context-aware voice design for social robots in public spaces.

Multimodal emotion recognition (MER) aims to identify human emotions from inputs such as text, vision, and audio. However, existing methods often assume complete modality availability during training and inference, which is unrealistic in real-world scenarios due to sensor failures or privacy constraints. We propose Dual-Query Fusion (DQF), a framework that enables robust MER using only incomplete modality inputs, without relying on reconstruction or knowledge distillation. DQF introduces two types of learnable queries: Q-UA for extracting informative unimodal features, and Q-CA for adaptive cross-modal integration. These modules are designed to operate effectively even when some modalities are missing. Experiments on two public datasets demonstrate that DQF achieves superior performance and robustness compared to existing methods, even when trained exclusively on incomplete inputs. These results highlight the effectiveness and practicality of DQF for real-world MER tasks.

In this paper, we aim to understand how user motivation shapes human-robot interaction (HRI) in the wild. To explore this, we conducted a field study by deploying a fully autonomous conversational robot in a shopping mall over two days. Through sequential video analysis, we identified five patterns of interaction fluency (Smooth, Awkward, Active, Messy, and Quiet), four types of user motivation for interacting with the robot (Function, Experiment, Curiosity, and Education), and user positioning towards the robot. We further analyzed how these motivations and positioning influence interaction fluency. Our findings suggest that incorporating users' motivation types into the design of robot behavior can enhance interaction fluency, engagement, and user satisfaction in real-world HRI scenarios.

Turn-taking is a crucial aspect of human-robot interaction, directly influencing conversational fluidity and user engagement. While previous research has explored turn-taking models in controlled environments, their robustness in real-world settings remains underexplored. In this study, we propose a noise-robust voice activity projection (VAP) model, based on a Transformer architecture, to enhance real-time turn-taking in dialogue robots. To evaluate the effectiveness of the proposed system, we conducted a field experiment in a shopping mall, comparing the VAP system with a conventional cloud-based speech recognition system. Our analysis covered both subjective user evaluations and objective behavioral analysis. The results showed that the proposed system significantly reduced response latency, leading to a more natural conversation where both the robot and users responded faster. The subjective evaluations suggested that faster responses contribute to a better interaction experience.

In recent years, the demand for social robots has grown, requiring them to adapt their behaviors based on users’ states. Accurately assessing user experience (UX) in human-robot interaction (HRI) is crucial for achieving this adaptability. UX is a multi-faceted measure encompassing aspects such as sentiment and engagement, yet existing methods often focus on these individually. This study proposes a UX estimation method for HRI by leveraging multimodal social signals. We construct a UX dataset and develop a Transformer-based model that utilizes facial expressions and voice for estimation. Unlike conventional models that rely on momentary observations, our approach captures both short- and long-term interaction patterns using a multi-instance learning framework. This enables the model to capture temporal dynamics in UX, providing a more holistic representation. Experimental results demonstrate that our method outperforms third-party human evaluators in UX estimation.

The rapid development of artificial intelligence (AI) has fundamentally transformed creative work practices in the design industry. Existing studies have identified both opportunities and challenges for creative practitioners in their collaboration with generative AI and explored ways to facilitate effective human-AI co-creation. However, there is still a limited understanding of designers' collaboration with AI that supports creative processes distinct from generative AI. To address these gaps, this study focuses on understanding designers' collaboration with decision-making AI, which supports the convergence process in the creative workflow, as opposed to the divergent process supported by generative AI. Specifically, we conducted a case study at an online advertising design company to explore how professional graphic designers at the company perceive the impact of decision-making AI on their creative work practices. The case company incorporated an AI system that predicts the effectiveness of advertising design into the design workflow as a decision-making support tool. Findings from interviews with 12 designers identified how designers trust and rely on AI, its perceived benefits and challenges, and their strategies for navigating the challenges. Based on the findings, we discuss design recommendations for integrating decision-making AI into the creative design workflow.

Promotional media, such as paper posters and digital signage, are installed in shopping malls to recommend products and services. However, it has been reported that many customers tend not to be interested in these promotional media and do not receive the information. When product information is not communicated effectively, advertisers are unable to convey the information they wish to share with customers, and customers miss the opportunity to receive valuable information. To address such issues, a lot of methods have been proposed to make people aware of the presence of media; however, there are not many methods that take into account the delivery of product information to customers. In this study, we propose Popping-Up Poster, a pin-based poster device designed to capture customer attention and convey information through dynamic shape changes. To verify the effectiveness of the proposed system, field experiments were conducted in a café, where its promotional effects were compared with those of traditional promotional media, including paper posters and digital signage. These results show that Popping-Up Poster has the potential to be more effective in recommending products and influencing customer product choices compared to conventional promotional media.

Service robots have increasingly been investigated in retailing. Previous studies mainly focused on the effectiveness of recommendation with regard to a single robot, and whether and how the use of two robots combined can achieve better performance remain unclear. In this study, we address this by exploring the combination power of two service robots for product recommendation in a bakery. We placed one robot inside the store for product recommendation and the other robot outside to promote the inside robot. Particularly, we are interested in the effects of the outside robot on the inside robot's performance in product recommendation. Our results indicate that using the outside robot to promote the inside robot achieved more purchases over using the inside robot alone. Particularly, we discovered that the outside robot increased the attention of customers toward the inside robot; hence, more customers checked and purchased the products. Based on the findings, we discuss the important points for the effective use of service robots.

Service robots are beginning to be used as a new kind of support for human labor. However, in many cases, we implement only specific task-dependent behaviors in robots according to the purpose of robot introduction, and rarely implement task-independent behaviors. In general, it is known that noninstrumental functions are one factor that improves user experience. Therefore, task-independent behavior of robots as an aspect of noninstrumental functions also has the potential to improve the impression made by robots and deliver a user experience beyond the users’ expectations, during human-robot interaction. This study aims to investigate the influence of task-dependent behavior and task-independent behavior on the impression made by robots, and user experience. We extracted, from previous studies, dialogue task-dependent and dialogue task-independent behaviors during human-robot interaction, and investigated the influence of these behaviors through a video-based survey. The result of the video-based survey shows that dialogue task-dependent behavior improves the functionality of robots and decreases factors of negative user experience, such as frustration, while also fulfilling users’ expectation for interaction with robots. It also shows that dialogue task-independent behavior builds a stronger relationship between users and robots and provides a user experience that exceeds users’ expectation regarding interaction with robots.

In recent years, communication robots have been employed to assist workers. However, it is known that users experience embarrassment when interacting with a robot in a public space, which may hinder their use. Previous studies investigated methods to reduce embarrassment when using robots and the factors that cause embarrassment. Although these studies have investigated the embarrassment experienced by users through only a single interaction, they have not investigated the embarrassment influenced by users’ past experiences. Therefore, in this study, we investigated changes in embarrassment and the factors causing embarrassment through repeated interactions with robots in public spaces. We conducted experiments in which the same participants used a robot in a public space multiple times and continuously experienced embarrassment through repeated interactions with the robot. The results show that repeated interactions with the robot reduce embarrassment, and that embarrassment is influenced by two factors: understanding of the user’s behavior from surrounding people and the user’s previous experience with the interaction.

For a humanoid robot to make eye contact and initiate communication with a person, it is necessary to estimate the person's head position. However, eye contact becomes difficult due to the mechanical delay of the robot when the person is moving. Owing to these issues, it is important to conduct a head-position prediction to mitigate the effect of the delay in the robot motion. Based on the fact that humans turn their heads before changing direction while walking, we hypothesized that the accuracy of three-dimensional (3D) head-position prediction from a first-person view can be improved by considering the head pose. We compared our method with a conventional Kalman filter-based approach, and found our method to be more accurate. The experiment results show that considering the head pose helps improve the accuracy of 3D head-position prediction.

Social robots are used in daily life. One of the applications of social robots is as recommendation systems. Previous research has mainly investigated how persuasive recommendations can be improved by focusing on the non-verbal/verbal behavior of robots. However, to use robots as recommendation systems every day, it is extremely important to examine the persistence of repeated persuasion over a long term, rather than the effect of one-time persuasion. Therefore, the objective of this study was to investigate the persistence of repeated persuasive of robots. For this purpose, robots with three types of behavior (Expert Behavior, Local Behavior, and Growth Behavior) recommended nutrition bars in a situation of daily consumption behavior for two weeks. We could confirm significant differences in the persistent persuasiveness in each behavior. The results suggested that the combination of value co-creation using local information and meta-trust expression had a significant impact on the persistence of the repeated persuasiveness of the robots in the longitudinal period. However, the acceptance of the recommendation robot system decreased due to the increase in the amount of information during recommendation; therefore, a new recommend system to solve this problem is desired.

In this study, we aimed to verify whether expanding a robot’s embodiment influenced the ease of inducing behavioral changes in passersby. We conducted a field experiment to call out to passersby for disinfection in a real shop using robots with various embodiments. As a result, the expansion of the robot embodiment attracted extensive attention of passersby, and then induced behavioral changes to the desired action, such as stopping in front of the robot and disinfecting their hands. Furthermore, the results revealed that in order to induce large behavioral changes, not only the presence of the robot’s body but also its movements with strong embodiment were necessary. These results indicated that expanding the robot’s embodiment was an important factor in inducing behavioral changes in passersby.

Understanding the relationship between driving behavior and visual information is an important issue in order to understand driving behavior holistically. In this study, we constructed a driver model that reproduces the driver's steering behavior from visual information based on the Convolutional Neural Network (CNN) with human physical characteristics. We obtained the driving behavior in a simulator study to train the proposed CNN model. Which region in the visual field influencing drivers' steering behavior was analyzed using the results of the feature maps generated by the trained CNN model and the driver's gaze behavior. The results indicate that the drivers perform steering action using the information within 20 degrees from the gaze point.

A concern has been raised regarding the possible increase of motion sickness in automated vehicles. Therefore, automated vehicles without motion sickness must be developed to provide a comfortable space for drivers. In this study, we propose a control method of automated vehicles that reduces the incidence of motion sickness in passengers. In the proposed method, the desired trajectory for the lane-change task and the steering-controller improvements in a path-following controller to track the path are determined to minimize motion sickness, based on a computational model of motion sickness. Numerical simulation results demonstrate that the vehicle can be successfully controlled during the lane-change task. The effectiveness of the proposed method is demonstrated by comparing with comfort indices of previous studies.

Interest for motion sickness is increasing with increasing opportunities to view digital devices in transportation systems, including automated vehicles. Hence, technology for predicting and estimating motion sickness is essential. As one such technology, computational models for estimating motion sickness from head movements have been proposed and are used for motion sickness evaluation. However, a model capable of handling the effects of visual input and the visual-vestibular interaction for motions with six-degrees-of-freedom has yet to be developed. In this study, therefore, a computational model of motion sickness with visual-vestibular inputs is proposed by extending a model of the subjective vertical conflict theory of motion sickness for a vestibular input only. The proposed model inputs are the acceleration and angular velocity of the head and the visual perception of the angular velocity. A simulation conducted by inputting 1 h of sinusoidal oscillations demonstrated that the results are consistent with those of similar experiments conducted with human participants. In addition, the results of a simulation experiment conducted by changing the visual input demonstrated that motion sickness increases significantly when a conflict occurs between the visual and vestibular signals, which imitates the reading of a book in a moving car. Furthermore, we developed a method to calculate the predicted MSI using experimental data measured by IMU and camera image by approximating the visual perception of the angular velocity through an optical flow analysis. The results of inputting camera images and inertial measurement unit signals obtained by sine-wave-like pitching motion into the proposed model demonstrate that the proposed method, can describe the difference in motion sickness by the changes in the visual environment.

It is an important functional behavior for humanoid robots to have face-to-face contact with humans. We predict future face position to achieve natural behavior that is similar to the communication between people. Robots gaze at a prediction point for reducing mechanical delay. The proposed system for robots to have face-to-face contact can reduce delay.